Housing Prices Regression Kaggle Competition¶

Date Created: Sunday 19th April 2020
Author: Scott Jenkins
Contact: https://www.linkedin.com/in/sjenkins97/
Date Modified: Saturday 16th May 2020

Brief:¶

In this notebook I apply linear regression models to predict house prices in this Kaggle competition: https://www.kaggle.com/c/house-prices-advanced-regression-techniques/overview|

Ask a home buyer to describe their dream house, and they probably won't begin with the height of the basement ceiling or the proximity to an east-west railroad. But this playground competition's dataset proves that much more influences price negotiations than the number of bedrooms or a white-picket fence.

With 79 explanatory variables describing (almost) every aspect of residential homes in Ames, Iowa, this competition challenges you to predict the final price of each home.

Contents:¶

Importing Libraries
Importing Data
Cleaning
Model 1 & 2: Univariate/Multivariate Linear Regression
Model 3,4 & 5: Regularised Linear Regression
Submission Export

Importing Libraries ¶

import pandas as pd
import numpy as np
import statistics

# Ignore Warnings
import warnings
warnings.filterwarnings("ignore")

# View All Columns
from IPython.display import display
pd.options.display.max_columns = None

# Visualisation
import seaborn as sns
import matplotlib.pyplot as plt
%matplotlib inline
import plotly.express as px
import plotly.graph_objects as go
import plotly.offline as pyo
pyo.init_notebook_mode()
from simple_colors import *

# Linear Regression
from sklearn import preprocessing, metrics
from sklearn.metrics import mean_squared_error, r2_score
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression, Ridge, Lasso, ElasticNet
import statsmodels.formula.api as smf

Importing Data ¶

This dataset details the house prices of residential properties in Ames, Iowa, United States. There are 79 explanatory variables for condsideration.

Train contains 1460 properties; Test contains 1459 properties the sale prices of which I aim to predict.

train = pd.read_csv(r'C:\Users\sjenkins\OneDrive - Dunelm (Soft Furnishings) Ltd\Documents\Side_Projects\House_Price_Regression\train.csv')
test = pd.read_csv(r'C:\Users\sjenkins\OneDrive - Dunelm (Soft Furnishings) Ltd\Documents\Side_Projects\House_Price_Regression\test.csv')

print('Train has', train.shape[0], 'rows and', train.shape[1], 'columns.')
print('Test has' , test.shape[0], 'rows and', test.shape[1], 'columns.')

Train has 1460 rows and 81 columns.
Test has 1459 rows and 80 columns.

train.head()

test.head()

Data Cleaning ¶

Before building models, it is necessary to clean our data - removing outliers, handling missing values, and removing collinear features.

Outline Approach:¶

Check for and remove any outliers from training set.

Check for skew and transform target variable (House Price) to have Normal Distribution.
Concatenate Train and Test sets to create a single frame to clean. Note that this is the easiest and quickest way to ensure that my train and test sets have the same features after cleaning.
Handle missing values through dropping columns or data imputation as appropriate.
Check for correlation between independent variables and remove any with high (> 0.9) collinearity.
Consider binning categorical variables to create more helpful features.
Apply one-hot encoding on categorical variables.
Split clean frame back into train and test splits.

It is good practice to scale numerical features to obey a Gaussian (0,1). This is often performed as part of model building.

1. Check for and remove any outliers from training set.¶

train.describe().T

First, I'd like to check the distribution of Lot Area. The max is much higher than the upper quartile.

plt.subplots(figsize = (16, 8));
sns.scatterplot(x = 'LotArea',y='SalePrice',data=train).set_title("Lot Area vs Sale Price",size=16);

I imagine the properties with the largest lot areas are farms with lots of land. The largest Lot Area is the test set is 56600. I choose to drop properties from the train set with a lot area above this.

test['LotArea'].max() # 56600

56600

train = train[train['LotArea']<56600]
print('Remaining Properties in Training Set:',train.shape[0])

Remaining Properties in Training Set: 1453

Next, I'd like to check the Ground Living Area, again the max is far higher than the upper quartile.

plt.subplots(figsize = (16, 8));
sns.scatterplot(x = 'GrLivArea',y='SalePrice',data=train).set_title("Ground Floor Living Area vs Sale Price",size=16);

There appears to be a fairly strong linear relation, except for the point in the bottom right corner. I declare this an outlier and drop it from my test set.

train[['GrLivArea','SalePrice']].sort_values('GrLivArea',ascending=False).head(1) # The Outlier!

train = train[(train['GrLivArea'] < 4600)]
print('Remaining Properties in Training Set:',train.shape[0])

Remaining Properties in Training Set: 1452

I am happy to move on to the next step, though there may be further outliers to remove when working on improving the models.

2. Check for skew and transform target variable (House Price) to have Normal Distribution.¶

This shouldn't actually be necessary thanks to the Central Limit Theorem. One to check if this has any impact on model accuracy.

plt.subplots(figsize = (16, 8));
sns.distplot(train['SalePrice']).set_title("Distribution of House Sale Prices",size=16);

plt.subplots(figsize = (16, 3));
fig = sns.boxplot(train['SalePrice']).set_title('Boxplot of House Sale Prices',size=16);

print("Skewness in SalePrice :", train['SalePrice'].skew())
print("Kurtosis in SalePrice :", train['SalePrice'].kurt())

Skewness in SalePrice : 1.8991383834139053
Kurtosis in SalePrice : 6.642615528379048

Apply a log transform to reduce right skew.

train['LogSalePrice'] = np.log(train['SalePrice']) # +1 if sale price < 1, else log is negative!

plt.subplots(figsize = (16, 8));
sns.distplot(train['LogSalePrice']).set_title("Distribution of Log-Transformed House Sale Prices",size=16);

print("Skewness in SalePrice :", train['LogSalePrice'].skew())
print("Kurtosis in SalePrice :", train['LogSalePrice'].kurt())

Skewness in SalePrice : 0.12530938026189326
Kurtosis in SalePrice : 0.826278184626108

3. Concatenate Train and Test sets to create a single frame to clean.¶

Note that this is the easiest and quickest way to ensure that my train and test sets have the same features after cleaning.

split_row = len(train)
y_train = train['LogSalePrice']
data = pd.concat([train, test],ignore_index=True, sort=False)
data

Dropping the target and Id columns.

data.drop(['SalePrice','LogSalePrice'],axis=1,inplace=True) #'Id'

4. Handle missing values through dropping columns or data imputation as appropriate.¶

First, a visual, with the null values in green. We see the features Alley, Fireplace Quality, Pool, Fence and Misc Feature each have lots of missing values.

plt.subplots(figsize = (16, 8));
sns.heatmap(data.isnull(), yticklabels = False, cbar = False, cmap ='Greens');

missing = pd.DataFrame(data.isnull().sum().reset_index()).rename({'index':'Feature',0:'Missing Value Count'},axis=1)
missing['Missing Value %'] = missing['Missing Value Count'] / len(data)
missing.sort_values('Missing Value %',ascending=False).head(34)

This next part will be a bit of a slog. The aim is to tackle all 34 of the features with missing values, so that I have a complete dataset.

First, check the documentation which describes the features in the dataset. Notice that some features are null if the property does not have this feature. Pool and Garage are two examples. I will fill the null values with 'None'.

fill_with_none = ['PoolQC', 'MiscFeature', 'Alley', 'Fence', 'FireplaceQu', 'GarageCond',
       'GarageQual', 'GarageFinish', 'GarageType', 'BsmtCond', 'BsmtExposure',
       'BsmtQual', 'BsmtFinType2', 'BsmtFinType1', 'MasVnrType']

for column in fill_with_none:
    data[column].fillna('None',inplace =True)

Next, I choose to fill categorical variables with the modal value.

fill_with_mode = ['MSZoning','Functional', 'Utilities', 'Electrical', 'KitchenQual', 'SaleType',
       'Exterior2nd', 'Exterior1st']

for column in fill_with_mode:
    data[column].fillna(data[column].mode()[0],inplace=True)

Now, I choose to fill continuous variables with the median value.

Lot Frontage is an interesting one. There are enough missing values that could make it worthwhile exploring this further. One can imagine that this is function of LotArea, LotShape and LotConfig. Instead of imputing with the median, I could run regression of the named variables to predict the LotFrontage values for these 400 + properties.

This may be worth looking at when improving the model.

fill_with_median = ['GarageYrBlt', 'GarageCars', 'GarageArea', 'LotFrontage']

for column in fill_with_median:
    data[column].fillna(statistics.median(data[column]),inplace=True)

What's left?

missing = pd.DataFrame(data.isnull().sum().reset_index()).rename({'index':'Feature',0:'Missing Value Count'},axis=1)
missing['Missing Value %'] = missing['Missing Value Count'] / len(data)
missing[missing['Missing Value %']>0].sort_values('Missing Value %', ascending=False)
#list(missing[missing['Missing Value %']>0]['Feature'])

I choose to fill missing values from the remaining features with 0, based on their absence of Basement or Masonry Fireplace.

fill_with_0 = ['BsmtFinSF1', 'BsmtFinSF2', 'BsmtUnfSF', 'TotalBsmtSF', 'BsmtFullBath','BsmtHalfBath','MasVnrArea']

for column in fill_with_0:
    data[column].fillna(0,inplace=True)

What's left? Nothing, I have handled them all.

plt.subplots(figsize = (16, 8));
sns.heatmap(data.isnull(), yticklabels = False, cbar = False, cmap ='Greens');

missing = pd.DataFrame(data.isnull().sum().reset_index()).rename({'index':'Feature',0:'Missing Value Count'},axis=1)
missing['Missing Value %'] = missing['Missing Value Count'] / len(data)
missing[missing['Missing Value %']>0].sort_values('Missing Value %', ascending=False)

All Green -> No Null Values

5. Check for correlation between independent variables and remove any with high (> 0.9) collinearity.¶

f, ax = plt.subplots(figsize=(14, 10))
sns.heatmap(data.corr().abs(), 
        mask=np.triu(np.ones_like(data.corr().abs(), dtype=np.bool)),
        square=True, 
        linewidths=.1,
        center = 0.5,
        cmap='RdYlGn_r').set_title('Heatmap of All Features',size=16);

# Select upper triangle of correlation matrix
upper = data.corr().abs().where(np.triu(np.ones(data.corr().shape), k=1).astype(np.bool))

# Find index of feature columns with correlation greater than 0.9
to_drop = [column for column in upper.columns if any(upper[column] > 0.9)]

print('Drop Columns:',to_drop)


# Drop features 
data.drop(data[to_drop], axis=1,inplace=True)

Drop Columns: []

One might expect Year Built and Garage Year Built to exhibit stronger correlation. I'd like to take a look at the distributions of these.

plt.subplots(figsize = (16, 8));
sns.distplot(data['YearBuilt']).set_title("Distribution of House Year Built",size=16);

plt.subplots(figsize = (16, 3));
fig = sns.boxplot(data['YearBuilt']).set_title('Boxplot of House Year Built',size=16);

plt.subplots(figsize = (16, 8));
sns.distplot(data['GarageYrBlt']).set_title("Distribution of Garage Year Built",size=16);

sns.boxplot(data['GarageYrBlt']).set_title("Distribution of Garage Year Built",size=16);

Aha, an outlier! Clearly the garage was built in 2007 instead of 2207!

data.at[2584, 'GarageYrBlt'] = 2007 # Setting this value in the dataframe.

6. Consider binning categorical variables to create more helpful features.¶

E.g. Overall Quality could be Great, Okay or Poor to cut down 10 features to 3.

I choose not to do this now, and leave this as something which could further improve later models.

7. Apply one-hot encoding on categorical variables.¶

First transform numeric variables into categorical variables if their numerical value doesn't matter. For example, the month of sale is encoded between 1 and 12, but it doesn't make sense for the model to consider this.

change_to_string = ['MSSubClass','YearBuilt', 'YearRemodAdd','GarageYrBlt','MoSold','YrSold']

for column in change_to_string:
    data[column] = data[column].apply(str)

data = pd.get_dummies(data,drop_first=True) # Drop First = True removes issues with linear dependence later on.

data

8. Split clean frame back into train and test splits.¶

train = data[:split_row]  #Recall that we set split_row = len(train) when concatenating
test = data[split_row:]
train
test

Join back on the House Prices for the training data.

log_prices = pd.DataFrame(y_train).reset_index().rename({'index':'Id'},axis=1)
log_prices['Id'] = log_prices['Id']+1
train = train.merge(log_prices)
train

Model 1: Univariate Linear Regression: Ground Floor Area to Predict Sale Price ¶

One can imagine that the Size of the House will be related to the Price. As a first attempt, let's focus on a single variable. Build a model with all of the training data, apply this to the test data and submit to put a mark on the leaderboard.

px.scatter(train,x='GrLivArea',y='LogSalePrice',title='Ground Floor Area vs. Log Sale Price')

model = LinearRegression(normalize = True)  
model.fit(train[['GrLivArea']], y_train) #training the algorithm
print('Model Intercept:',round(model.intercept_,2))
print('Model Slope:', round(model.coef_[0],2))

LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None, normalize=True)

Model Intercept: 11.16
Model Slope: 0.0

np.exp(model.predict(test[['GrLivArea']])) # Out put the predictions for the test set.

array([117365.94367828, 150179.6228743 , 178152.82165401, ...,
       141464.48271672, 122416.58124591, 220054.74408825])

test['Log_Predictions'] = model.predict(test[['GrLivArea']])
test['Predictions'] = np.exp(test['Log_Predictions'])

test

fig = go.Figure(data=go.Scatter(
                x=train['GrLivArea'],
                y=train['LogSalePrice'],
                mode='markers',
                name = 'Data',
                hovertemplate='Living Area: %{x} <br>Log Sale Price: %{y} '
));

fig.add_trace(go.Scatter(
                x=test['GrLivArea'],
                y=test['Log_Predictions'],
                mode='markers',
                name = 'Predictions',
                hovertemplate='Living Area: %{x} <br>Log Prediction: %{y} '
));

fig.update_layout(
    title='Univariate Regression Line: Ground Floor Size vs. Sale Price',
    xaxis_title='Ground Floor Living Area/(Sq.ft)',
    yaxis_title='Log Sale Price/$'
)

Result

An Improvement of a 0.40890 RMSE to a 0.29403 RMSE. This pushed me into 3,800th up from 4200th.

Model 2: Multivariate Linear Regression¶

I'm going to split my test set into a training set and a validation set with a 75:25 split.

I will then build my models with the training set data, and will then assess each models performance on the validation set. With this method, I can have a fairer view on model performance, and on the bias / variance trade-off.

I can explore the impact of changing the model features, changing the regularisation parameter, and even experimenting with neural networks. and have a fairer view on model performance.

Then, the best models can be used to predict the house prices in the test set.

train

X_train, X_val, y_train, y_val = train_test_split(train.drop(['Id','LogSalePrice'],axis=1), train['LogSalePrice'], test_size=0.25, random_state=42)

model = LinearRegression(normalize = True,)  
model.fit(X_train, y_train) # Fit the model to the training data
print('Model Intercept:',round(model.intercept_,2))
print('Model Coefficients:', model.coef_)

LinearRegression(copy_X=True, fit_intercept=True, n_jobs=None, normalize=True)

Model Intercept: -16717754167.21
Model Coefficients: [ 4.20015089e-04  3.93673720e-06  2.00961736e-02  5.05119510e-02
  2.95032069e-05  9.82337781e+07  9.82337781e+07  9.82337781e+07
 -9.82337781e+07  8.88878096e+07  8.88878096e+07  8.88878096e+07
 -8.88878096e+07  2.51853417e-02  3.19579095e-02  3.29309966e-02
  1.79235893e-02 -1.15175877e-02  3.26693460e-02  5.09981116e-03
  3.14197421e-02  3.34852713e-02  6.32292831e-05  9.62348008e-05
  1.42507662e-04  1.89860775e-06  1.11494049e-04  2.80854332e-04
  1.54117986e-03 -3.99133581e-05 -4.00135520e+11 -7.64246394e-02
 -4.69600611e-02  6.40282267e-01  1.14246340e-01  9.01302849e-02
  2.43773508e-01  2.23828366e-01  2.63553264e-01  6.18739008e-02
  1.16932416e-01  6.21165659e-02  1.89693976e-02  1.20632107e-01
  1.02009010e+11  5.70044527e-01  4.95876410e-01  5.23047251e-01
  4.27567689e-01  1.05991652e-01 -2.66533322e-02 -3.98267292e-02
  4.98902984e-03  2.47967197e-02  2.78045708e-02  4.64766261e-02
  2.83130207e-02  2.44596578e-02 -3.85056900e-01  3.72306159e-02
 -4.46986564e-02 -5.90885881e-02 -1.30654744e-02  2.59142489e-02
 -2.49152676e-01 -5.83077222e-02  5.67836833e-02  5.56351627e-02
  4.44168810e-02 -7.89404084e-03  1.26083414e-01 -3.92896176e-02
 -1.21302102e-02  9.76372118e-02 -2.92143956e-02 -3.84212796e-02
 -1.13111109e-02  2.68956306e-02 -3.56589239e-02  5.54971736e-02
  8.00110406e-02  1.67832989e-02 -2.30663167e-02  2.90417041e-03
  2.76684712e-03  2.43590026e-02  1.30303257e-01  3.11626928e-02
  1.84100550e-02  1.08141067e-01  1.28734760e-01  1.27305530e-01
  1.69627917e-01  1.23997135e-03  1.22281526e-01  1.23725491e-01
  1.51875252e-01  3.49488495e-01  1.85970584e-01  1.12656026e+11
  3.54929844e-04 -3.84179117e-01  5.16300302e+09 -3.05759056e+10
 -5.70623243e-01 -1.02009010e+11  7.89602662e-02  8.37038264e-02
  4.09512279e-02  9.89504346e-02 -4.28517213e-03  3.59223491e-01
  1.34503406e-01  8.52717370e-02  2.12037434e-01  8.48018310e+09
 -2.42703031e+10  9.21821602e+09  9.21821602e+09  9.21821602e+09
  9.21821602e+09  9.21821602e+09 -1.03437810e+11 -2.28130443e+10
 -2.05484172e+10  9.21821602e+09  9.21821602e+09  1.81602874e+10
 -1.59979765e+09  9.21821602e+09  9.21821602e+09 -1.55100661e+10
 -1.59948737e+10 -2.15933756e+10  9.21821602e+09  9.21821602e+09
  9.21821602e+09  9.21821602e+09  9.21821602e+09  9.21821602e+09
  9.21821602e+09 -8.96509829e+09  9.21821602e+09  9.21821602e+09
  9.21821602e+09  9.21821602e+09  9.21821602e+09  9.21821602e+09
  9.21821602e+09  9.21821602e+09  9.21821602e+09 -5.90424202e+10
 -5.90424202e+10  9.21821602e+09  9.21821602e+09  9.21821602e+09
  9.21821602e+09  9.21821602e+09  9.21821602e+09  9.21821602e+09
  9.21821602e+09  9.21821602e+09  9.21821602e+09  9.21821602e+09
  9.21821602e+09 -6.62058226e+10  9.21821602e+09  9.21821602e+09
  9.21821602e+09  9.21821602e+09  9.21821602e+09  9.21821602e+09
  9.21821602e+09  2.42138200e+10  9.21821602e+09  9.21821602e+09
  9.21821602e+09  9.21821602e+09  9.21821602e+09  9.21821602e+09
  9.21821602e+09  9.21821602e+09  9.21821602e+09  9.21821602e+09
  9.21821602e+09  9.21821602e+09  9.21821602e+09  9.21821602e+09
  9.21821602e+09  9.21821602e+09  9.21821602e+09  9.21821602e+09
  9.21821602e+09  9.21821602e+09  3.86485592e+10  9.21821602e+09
  9.21821602e+09  9.21821602e+09  9.21821602e+09  9.21821602e+09
  9.21821602e+09  9.21821602e+09  9.21821602e+09  9.21821602e+09
  9.21821602e+09  9.21821602e+09  9.21821602e+09  9.21821602e+09
  9.21821602e+09  9.21821602e+09  9.21821602e+09  9.21821602e+09
  9.21821602e+09  9.21821602e+09  9.21821602e+09  9.21821602e+09
  9.21821602e+09  9.21821602e+09  9.21821602e+09  9.21821602e+09
  9.21821602e+09  9.21821602e+09 -1.93162087e+10  9.21821602e+09
  9.21821602e+09  9.21821602e+09  9.21821602e+09  9.21821602e+09
  9.21821602e+09  9.21821602e+09  9.21821602e+09  9.21821602e+09
  3.86925272e-02 -1.49956040e+10  6.05744957e-02  1.18793421e-02
  4.32230188e-02 -7.13228608e-02 -2.29816178e-02  1.09942685e-02
 -5.18339765e-02  4.25380565e-02  1.07200667e-01  1.76722685e-01
 -5.60694469e-03  2.73632843e-01  8.64060425e-03  1.62981062e-01
  2.32537503e-02  9.27279859e-02 -8.99979238e-02  1.10959844e-01
  1.29996764e-01  1.14983559e-02  1.01461062e-01 -4.69542500e-02
 -1.15487263e-01  1.94962732e-01 -4.16734747e-03  3.63907226e-01
 -2.41443838e-01 -4.51647807e-02 -1.76223019e-01  8.09497749e-02
  5.12495550e-02  2.37995711e-02 -4.70931672e-02 -3.22553379e-02
 -3.49857709e-02  8.70200233e-02 -5.26472010e-02 -2.36914740e-02
  1.50930995e-02  2.65738637e-02  2.24346267e-02  3.38622641e-02
  5.32477022e-02  5.27326351e-03 -2.87345536e-02  7.55307132e-03
  2.35558683e-02  1.56018542e-02  5.71440374e-02  5.30430488e-02
  8.67769126e-03 -8.56313585e-03 -2.41330567e-02 -1.98804027e-02
  3.84486874e-02  4.48316766e-02  2.70949274e-02  8.45915466e-02
 -1.33792220e-01 -1.52859728e-01 -1.33359574e-01 -1.76288747e-01
  1.15317218e-01 -2.52032947e+07  1.51925239e-01 -2.10935188e-01
 -9.78120437e-02  5.65479023e-03 -6.12080966e-02 -3.25389171e+07
 -3.36448081e-01  1.49256663e-01 -7.74812134e+10  6.61322617e-02
  8.75549555e-02  2.06193123e+07  9.25147873e-02  5.37315122e-02
  2.66586661e-01  1.64869711e-01  5.86296694e-02  2.88763274e-02
  1.65366988e-01  1.93287924e-01  2.85592310e-02 -3.54827272e-02
  7.74812134e+10  3.42868990e-02 -2.71142265e-02  3.94945930e-02
 -6.50217858e-03 -1.51742050e-02 -4.75672467e-03 -1.51265869e-01
  2.89841578e-02  1.37738225e-02  3.47189356e-02 -8.06437405e-02
  8.62422026e-02  7.13924597e-02  1.01660473e-01  2.20987494e-01
 -3.35173809e-02 -2.76927637e-02  8.01261287e-03 -3.47925426e-03
 -5.70177621e-02  2.75403235e-02 -1.28421799e-02  2.02200886e-02
 -3.23857889e-02  1.78467188e-01 -1.19341657e-01  3.85650066e-02
 -2.03190179e-02  5.84916319e+10 -8.77306651e-03  4.04843599e-03
 -1.95137135e+10  1.91281004e-01  1.20147025e-02  3.55085379e-02
  1.77267611e-02  1.16701535e-03 -1.96580977e+10  1.29346873e-02
  1.16113125e-02 -8.80560304e-03 -1.93198207e+10 -6.24454027e-03
 -6.91294296e-03 -7.22830175e-02  1.94397642e-02 -4.16990804e-02
 -5.51051344e-02  1.31313887e-02  5.89006522e-04  3.83245513e-01
  3.88638065e-01  2.57028450e-01  9.21821602e+09  2.32599947e-01
  8.96259706e-03 -1.64844623e-02  3.93247002e-02 -1.17354621e-02
  9.86890061e-02 -2.59013182e-02 -1.56834416e-01  9.23886202e-02
 -1.82130375e-02 -9.25600575e-02 -4.28543053e-02 -5.28543005e-02
 -2.06024916e-01  6.59049782e-02  6.88342200e-02 -4.32989286e-03
 -2.52151998e+04  9.01790881e-02 -3.68964719e-02 -1.52078717e-02
 -9.47817267e-03 -4.31142139e-02 -2.97563889e-02  1.55267042e-01
  1.64980794e-01  1.56804738e-01 -2.61888982e-02  1.10300434e-01
  9.18413225e+10  4.36436517e+03  7.49953815e+09  3.22278202e+10
  3.83111298e+10  7.49953815e+09  7.49953815e+09  7.49953815e+09
  7.49953815e+09  7.72455404e+02  7.49953815e+09  9.21959914e+02
  7.49953815e+09  7.49953815e+09  7.49953815e+09  7.49953815e+09
  7.49953815e+09  7.49953815e+09  7.49953815e+09  7.57601743e+10
  7.57601743e+10  7.57601743e+10  7.49953815e+09  7.49953815e+09
  7.49953815e+09  7.49953815e+09  7.49953816e+09  7.49953815e+09
  7.49953815e+09  7.49953815e+09  7.49953815e+09  7.49953815e+09
  7.49953815e+09  7.49953815e+09  8.29235768e+10 -4.33805451e+01
  7.49953815e+09  7.49953815e+09  7.49953815e+09  7.49953815e+09
  7.49953815e+09  7.49953815e+09  7.49953815e+09  7.49953815e+09
  7.49953815e+09  7.49953815e+09  7.49953815e+09  7.49953815e+09
  7.49953815e+09  7.49953815e+09  7.49953815e+09  7.49953815e+09
  7.49953815e+09  7.49953815e+09  7.49953815e+09  7.49953815e+09
  7.49953815e+09  7.49953815e+09  7.49953815e+09  7.49953815e+09
  7.49953815e+09  7.49953815e+09  7.49953815e+09  7.49953815e+09
 -2.19308050e+10  7.49953815e+09  7.49953815e+09  7.49953815e+09
  7.49953815e+09  7.49953815e+09  7.49953815e+09  7.49953815e+09
  7.49953815e+09  7.49953815e+09  7.49953815e+09  7.49953815e+09
  7.49953815e+09  7.49953815e+09  7.49953815e+09  7.49953815e+09
  7.49953815e+09  7.49953815e+09  7.49953815e+09  7.49953815e+09
  7.49953815e+09  7.49953815e+09  7.49953815e+09  7.49953815e+09
  7.49953815e+09  7.49953815e+09  7.49953815e+09  7.49953815e+09
  3.60339629e+10  7.49953815e+09  7.49953815e+09  7.49953815e+09
  7.49953815e+09  7.49953815e+09  7.49953815e+09  7.49953815e+09
  7.49953815e+09  7.49953815e+09 -3.06137742e+10  5.86434656e-03
 -2.85667313e-02 -9.85930385e-01 -9.73731502e-01 -3.06137742e+10
 -1.25594567e+00 -9.88156041e-01  1.19257946e+00  9.89575041e-01
 -3.06137742e+10  1.23841474e+00  1.10285946e+00 -5.31111690e-02
  5.19243137e-02 -2.22113619e-01  1.21462142e-01  7.66483616e-01
 -2.40990533e-02  2.27686967e-02  2.72833945e-02  1.18707008e-02
 -6.26779410e-01 -8.64058371e-01 -6.36550459e-01 -5.40836754e-01
  4.58274239e-03  2.03237208e-02  3.20492994e-02  1.09721749e-02
  2.00800773e-02  2.79129539e-02  3.23722266e-02  1.62303718e-02
  2.63847895e-02  2.01482573e-02  3.06238946e-02 -1.32423315e-02
 -1.55074902e-02 -3.51557296e-02 -1.95174696e-02  1.03889704e-01
 -2.88676274e-02 -7.27577655e-03 -6.61310861e-02 -8.39308490e-02
  7.87402559e-02  5.24289439e-02 -6.73300515e-02  1.54152273e-01
 -1.84173270e-02  4.18122032e-03  7.12422072e-02 -5.24621501e-02]

train_results = pd.DataFrame(y_train)
train_results['Model_Output(Log)'] = model.predict(X_train)
print('Root Mean Squared Error:', np.sqrt(metrics.mean_squared_error(train_results['LogSalePrice'], train_results['Model_Output(Log)'])))

Root Mean Squared Error: 0.06738022470800617

train_results.head()

px.scatter(train_results,x='Model_Output(Log)',y='LogSalePrice',title='Multivariate Regression Training Predictions')

Testing the model on my validation set:

val_results = pd.DataFrame(y_val)
val_results['Model_Predictions(Log)'] = model.predict(X_val)
print('Root Mean Squared Error:', np.sqrt(metrics.mean_squared_error(val_results['LogSalePrice'], val_results['Model_Predictions(Log)'])))

Root Mean Squared Error: 9589855245.604158

px.scatter(val_results,x='Model_Predictions(Log)',y='LogSalePrice',title='Multivariate Regression Validation Predictions')

val_results

Result

An Improvement of a 0.29403 RMSE to a 0.28935 RMSE, with a bit of manual value imputing for the values predicted as 0 or infinity! This pushed me up 11 places, negligible. I know I can do better!

This appears to be overfitting, and I'm not surprised. I have more feature variables than houses in my validation set. More concerning are the 0 value predictions. One way to handle over-fitting (high variance) is through regularisation (penalising large parameter values). I will try this next.

Model 3: Introducing the Regularisation Parameter, Ridge, Alpha = 1 ¶

Ridge uses L2 penalties (square of the coefficients)

model = Ridge(alpha=1, fit_intercept=True)
model.fit(X_train, y_train) # Fit the model to the training data
print('Model Intercept:',round(model.intercept_,2))
print('Model Coefficients:', model.coef_)

Ridge(alpha=1, copy_X=True, fit_intercept=True, max_iter=None, normalize=False,
      random_state=None, solver='auto', tol=0.001)

Model Intercept: 10.16
Model Coefficients: [ 4.71227463e-04  4.42800956e-06  3.17560772e-02  4.78945122e-02
  6.97842888e-06  6.56376522e-05  5.26054297e-05 -3.96727545e-06
  1.14275798e-04  6.80840062e-05  8.50160739e-05 -9.49197818e-06
  1.43608112e-04  1.74920794e-02  7.12620319e-03  3.26849218e-02
  2.99960059e-02 -4.32684771e-03 -8.20690320e-02  5.97182722e-03
  2.32946877e-02  3.46294402e-02  6.35695211e-05  1.09085030e-04
  8.53659996e-05  7.68014906e-05  1.67543593e-04  2.34948874e-04
  1.84556140e-04 -9.89950907e-07  0.00000000e+00 -4.63411812e-02
  5.11729607e-03  2.92691715e-03  1.14263974e-02 -6.85391834e-02
  8.43927740e-02 -5.74682950e-04  4.85763780e-03 -8.94541831e-03
  1.44795302e-02  2.00837132e-02 -1.89123772e-02  3.71632790e-02
  3.65508551e-02  2.06995769e-01  1.84914379e-01  1.78813142e-01
  1.41546315e-01  6.62385611e-02 -2.93986538e-02 -1.10505614e-02
  2.04942749e-02  8.73136508e-03  1.85699646e-02  3.51008954e-02
  1.97904189e-02  2.95799989e-02 -7.40566066e-02  3.91246967e-02
 -4.52422308e-02 -4.44288271e-02 -7.01128552e-03  4.35877730e-05
 -1.19275262e-01 -3.02031349e-02  2.11611968e-03  2.35944068e-02
  3.03509712e-02 -8.10205000e-03  1.03121790e-01 -4.36314170e-02
 -5.50646830e-03 -4.01036152e-02 -9.71305615e-02 -3.11785742e-02
 -2.40168076e-02  4.63693890e-02 -2.86043250e-02  4.27352726e-02
  6.47690412e-02 -4.36774628e-02 -2.12422525e-02 -2.25485249e-02
  1.04652303e-02  2.69145454e-02  1.20393546e-01  1.29138168e-02
  1.63168715e-02  4.72294724e-02  6.79640311e-02  2.26574305e-02
  6.92210691e-02 -6.62492746e-02  4.62341134e-02  3.52467638e-02
  6.61460343e-02  7.70375972e-02  3.81141353e-02  4.52444639e-03
 -5.88113245e-02 -2.73202999e-02  0.00000000e+00  0.00000000e+00
 -2.55008963e-02  3.65508551e-02 -3.00978968e-02 -8.86526327e-03
 -4.13318955e-03 -9.26751370e-03 -2.95419034e-02  5.39941388e-02
 -8.13543917e-03 -4.11081289e-02  4.61772392e-02  0.00000000e+00
  0.00000000e+00 -5.86501173e-02 -2.60396146e-02 -5.62824518e-03
 -5.32361820e-02  1.18161069e-02  4.52444639e-03  0.00000000e+00
  0.00000000e+00 -5.97500350e-02 -6.34695643e-02  0.00000000e+00
  0.00000000e+00 -7.27764665e-03  3.78630643e-02 -2.20506062e-02
  0.00000000e+00 -2.71864403e-02 -7.96630690e-02  4.43920042e-02
 -3.54407978e-03 -6.19215904e-02 -4.80810534e-02 -2.74725477e-02
  4.83834471e-02  0.00000000e+00 -2.57457320e-02  7.15238328e-03
 -9.47828114e-02  4.37224423e-02 -6.85247006e-02  5.23956741e-02
 -5.07964909e-02 -2.83864119e-02 -5.86769974e-02 -1.14890422e-01
  9.43779659e-02  6.48057803e-04 -2.89708891e-02 -3.85978386e-02
  1.15313800e-01  2.57979495e-02 -4.61749388e-02  1.65235294e-02
 -3.72468225e-02  2.91015245e-02 -3.26383685e-02 -6.67660302e-03
 -3.92302739e-02 -7.95993613e-03  3.70970208e-02 -4.86665641e-02
 -1.99396202e-02 -2.55942899e-02  7.91216933e-03  5.72717174e-03
  9.54456921e-03  3.10911096e-02  3.00711628e-02  1.81591241e-02
  2.66135829e-02 -6.88526354e-03  2.20084531e-02  7.13925313e-02
 -3.41907759e-02  4.79370933e-02  2.18561040e-02 -1.06430971e-02
 -7.57249998e-03 -1.07847084e-01 -5.57036037e-02 -4.95225333e-02
  1.09510044e-02 -8.78418448e-02  2.99343796e-02 -3.90442771e-02
  8.89098459e-04 -5.17377605e-02 -4.36980917e-02  4.11827379e-02
 -1.21863693e-02 -5.27647582e-02  4.26716863e-02  1.76425355e-02
  1.33658670e-03  4.58896220e-02 -9.17746057e-03  3.96846031e-02
  1.29209760e-02 -1.54922924e-02  4.05043621e-02  2.80398100e-02
  7.19090268e-02  2.15845213e-03  5.40397736e-02  6.61614638e-02
 -3.92727774e-02 -9.36100697e-04 -1.85592921e-02 -2.39870596e-02
  5.11488592e-02  6.24890195e-02  6.27381905e-03  5.82148277e-02
  2.51896849e-02 -1.33588637e-03  1.54677115e-02  1.21391069e-02
  1.46954446e-02  3.90868688e-03  8.83510070e-02 -1.83535888e-04
  8.74806725e-02  1.21680437e-01  2.35054527e-02  8.06512898e-02
  1.88137335e-02  3.10911096e-02 -2.06210396e-02 -4.13204731e-03
  4.88097738e-02 -4.88104583e-02 -6.78721182e-03 -2.92982531e-02
 -3.48347902e-02 -7.90053595e-03  5.58623674e-02  1.15183248e-01
 -2.69851892e-02  3.39014586e-02 -2.68186125e-02  5.39649080e-02
  1.70575755e-02  5.66391763e-02 -4.30297396e-02  3.69390783e-02
  1.02224635e-02  3.77675781e-03  2.52659342e-02 -4.46880482e-03
 -4.31685544e-02  6.57520336e-02 -2.11868337e-02  3.47009904e-02
 -7.35421578e-02  3.83474709e-03 -3.26754061e-02  3.36624416e-02
  1.61179742e-02  3.67581974e-02 -3.62930916e-03  1.20645657e-02
  2.48802718e-02  4.38862118e-02 -3.23821434e-02 -1.73627046e-02
 -8.51855237e-03  2.16105030e-02  2.60076875e-02  3.66254143e-02
  2.31384380e-02  1.66407950e-02  9.43680623e-03  2.31548760e-02
  2.61792709e-02  2.78287289e-02  4.58861071e-02  3.53361917e-02
  2.72837469e-03 -1.12435337e-02 -1.76506567e-02 -2.27372647e-02
  5.20698826e-03  2.93481740e-02  3.48116765e-02  4.80075440e-02
 -2.85499225e-02  4.75428175e-03 -2.65650928e-02 -2.66181041e-02
  2.57721819e-02  0.00000000e+00  6.36094501e-02 -4.02141357e-02
 -2.38598861e-02  4.44798841e-02  1.61006619e-02  0.00000000e+00
 -1.17291901e-01  8.64472601e-02 -1.29143345e-02  1.69911353e-02
  2.45676177e-02  0.00000000e+00  3.71918266e-02  1.74927986e-02
  4.15023510e-02  2.76078968e-02  4.55388352e-02 -1.77771704e-02
  7.36022446e-02  5.20740678e-02 -4.17258228e-04 -1.01348192e-02
 -1.29143345e-02  5.01360913e-02  7.31402209e-03  3.64881152e-02
  9.99207000e-03 -2.20341253e-02  1.74459858e-02 -3.56676105e-02
  1.14426997e-02  4.69609453e-03  3.82716010e-02 -2.14440453e-02
  3.06438673e-02  2.06754102e-02  4.15048158e-02  4.48775251e-02
 -1.99136036e-02 -2.84288281e-02  1.51392940e-02 -3.01922330e-02
  1.37586838e-02 -9.39544727e-03  9.93398038e-04  3.74051155e-02
  4.64125346e-03  7.50842690e-03 -3.71508974e-02  5.42439717e-03
 -2.97853452e-02  8.92196866e-03 -3.80686889e-02  1.43977611e-02
  8.92196866e-03  5.76446002e-02  2.27876367e-02  3.16952890e-02
  7.00323365e-03  1.99301574e-04  8.92196866e-03  6.49831368e-03
  2.82590936e-03 -1.56728261e-02  8.92196866e-03 -2.16545912e-02
 -2.28374341e-02 -4.29481450e-02  4.11398185e-02 -5.22007378e-03
 -2.57328254e-02  3.74121102e-03  2.28123683e-02  7.93178710e-02
  8.05041649e-02 -6.19800595e-02 -3.24817577e-02  1.29557524e-03
  5.70227091e-04 -2.94835986e-02  2.23713347e-02 -2.74620681e-02
  8.00752173e-02  9.33475808e-03 -1.68546344e-02 -1.03106411e-02
 -1.48357524e-02 -4.60062695e-02 -3.93313213e-02 -4.72455329e-02
 -1.01100933e-01  4.23742437e-02  6.05761502e-02 -4.76346624e-02
  0.00000000e+00  9.06955285e-02 -3.61793076e-02 -5.50836868e-03
 -7.68303534e-03 -1.57413353e-02 -1.61268449e-02  6.59620970e-02
  2.80464845e-02  6.48563549e-02 -3.67129814e-02  3.88269369e-02
 -1.39429114e-03  0.00000000e+00 -1.81318167e-02 -2.20506062e-02
 -2.71864403e-02  3.18958083e-02 -2.01996965e-02 -4.08687093e-02
 -9.66154445e-02  0.00000000e+00 -4.30261844e-02  0.00000000e+00
  4.90021003e-02  2.35677631e-02  7.94330178e-02  8.11903327e-03
 -5.83058832e-02 -1.41377339e-02  4.67312683e-02  1.00689337e-01
 -9.55368646e-02 -2.56649293e-02  4.28368248e-02  3.75026751e-02
  4.04345379e-02  2.22588195e-02  3.52863981e-02  5.89132907e-02
  3.26950926e-02  4.74075081e-02 -1.14815317e-01 -1.55453727e-03
  1.37795730e-02 -6.91565660e-02 -7.95993613e-03  0.00000000e+00
 -8.17171042e-02 -1.11600993e-02  1.57703160e-02  1.78419276e-02
  8.16030736e-03 -2.48306616e-02  4.42584670e-02 -1.30252071e-01
 -5.88343244e-02 -1.58892100e-02 -5.42080053e-02  1.71380150e-02
 -5.09646932e-02 -1.05599734e-01  2.75380007e-02 -1.40944262e-02
 -9.73651202e-02 -1.59865901e-02 -1.10146820e-02  6.33472307e-02
  7.74479792e-03  5.89480012e-03 -4.50812495e-02  4.73876814e-02
 -1.21766663e-02  7.71214424e-03 -1.32726770e-02  3.26718518e-02
 -4.36980917e-02  2.88629869e-02  5.45491013e-03 -2.24386138e-02
 -7.28697945e-02 -4.03179678e-02  9.07835731e-02 -1.68990979e-02
  1.71899288e-02 -4.77063184e-02  1.27059745e-03 -2.57920202e-02
 -1.63909634e-02  2.29477449e-02  1.65424440e-02  2.16885913e-02
  4.41563747e-02 -6.80328074e-02  5.56196441e-02 -4.61927587e-02
  1.61951761e-02  3.92251798e-02  6.80366576e-03 -1.13954290e-02
  6.68218962e-02 -5.34367797e-03  2.32950731e-02  4.86120030e-02
  1.54677115e-02  7.55918308e-04  3.19998331e-02  7.17515737e-02
 -1.25622983e-02  9.55381868e-02 -7.18279690e-03  1.95675110e-02
  7.33179894e-02  5.45935144e-02 -1.39429114e-03  4.07752164e-03
 -1.40577777e-02 -6.17237749e-02 -1.22461592e-02 -1.39429114e-03
 -2.72061593e-02 -4.34033387e-02  1.47448912e-03 -3.48103003e-02
 -1.39429114e-03  4.97436161e-03 -3.70665374e-03 -3.13418631e-02
  4.66375191e-02 -7.25652339e-02  1.07525327e-01  3.41196635e-02
 -1.31332952e-02  8.92314351e-03 -1.19902497e-02  1.17185503e-03
  6.28803932e-02 -5.64693104e-02  4.72708846e-02 -5.47880966e-02
  2.47670396e-03  1.68471251e-03  1.78106762e-02  1.90685644e-03
  1.36692661e-02  1.26377550e-02  2.57177695e-02  8.04985869e-03
  1.63446731e-02  1.30913733e-02  4.45865264e-03 -8.51830678e-03
 -1.42621583e-02 -3.20364232e-02 -1.99569594e-02  3.67434198e-02
 -2.47031187e-03  4.57399642e-02 -4.16061553e-02 -2.88139929e-02
  9.35856346e-02  8.82376288e-02 -3.97731081e-02 -2.37815330e-03
  7.47353611e-03  3.31284294e-03  6.93317571e-02 -3.81155498e-02]

train_results = pd.DataFrame(y_train)
train_results['Model_Output(Log)'] = model.predict(X_train)
print('Root Mean Squared Error:', np.sqrt(metrics.mean_squared_error(train_results['LogSalePrice'], train_results['Model_Output(Log)'])))

Root Mean Squared Error: 0.07603728133760505

train_results.head()

px.scatter(train_results,x='Model_Output(Log)',y='LogSalePrice',title='Multivariate Regression Training Predictions')

Testing the model on my validation set:

val_results = pd.DataFrame(y_val)
val_results['Model_Predictions(Log)'] = model.predict(X_val)
print('Root Mean Squared Error:', np.sqrt(metrics.mean_squared_error(val_results['LogSalePrice'], val_results['Model_Predictions(Log)'])))

Root Mean Squared Error: 0.1259781162779618

px.scatter(val_results,x='Model_Predictions(Log)',y='LogSalePrice',title='Multivariate Regression Validation Predictions')

val_results

Result

An Improvement of a 0.28935 RMSE to a 0.13298 RMSE. This pushed me up over 2400 places, into position 1610. Next, I think experimenting with the alpha parameter in Ridge Regression will be fruitful.

Model 4: Adjusting the Alpha Parameter in Ridge Regression¶

alpha_vals = [0.01,0.02,0.04,0.08,0.16,0.32,0.64,1.28,2.56,5.12,10.24,20.48,40.96,81.92,163.84,327.68,655.36,1310,2621,5242,10485]
train_errors = []
validation_errors = []

for a in alpha_vals:
    model = Ridge(alpha=a, fit_intercept=True);
    model.fit(X_train, y_train); # Fit the model to the training data
    
    train_results = pd.DataFrame(y_train)
    train_results['Model_Output(Log)'] = model.predict(X_train)
    train_errors.append(np.sqrt(metrics.mean_squared_error(train_results['LogSalePrice'], train_results['Model_Output(Log)'])))
    #px.scatter(train_results,x='Model_Output(Log)',y='LogSalePrice',title='Train Output: Alpha = {}'.format(a))

    val_results = pd.DataFrame(y_val)
    val_results['Model_Predictions(Log)'] = model.predict(X_val)
    validation_errors.append(np.sqrt(metrics.mean_squared_error(val_results['LogSalePrice'], val_results['Model_Predictions(Log)'])))
    #px.scatter(val_results,x='Model_Predictions(Log)',y='LogSalePrice',title='Validation Output: Alpha = {}'.format(a))

Ridge(alpha=0.01, copy_X=True, fit_intercept=True, max_iter=None,
      normalize=False, random_state=None, solver='auto', tol=0.001)

Ridge(alpha=0.02, copy_X=True, fit_intercept=True, max_iter=None,
      normalize=False, random_state=None, solver='auto', tol=0.001)

Ridge(alpha=0.04, copy_X=True, fit_intercept=True, max_iter=None,
      normalize=False, random_state=None, solver='auto', tol=0.001)

Ridge(alpha=0.08, copy_X=True, fit_intercept=True, max_iter=None,
      normalize=False, random_state=None, solver='auto', tol=0.001)

Ridge(alpha=0.16, copy_X=True, fit_intercept=True, max_iter=None,
      normalize=False, random_state=None, solver='auto', tol=0.001)

Ridge(alpha=0.32, copy_X=True, fit_intercept=True, max_iter=None,
      normalize=False, random_state=None, solver='auto', tol=0.001)

Ridge(alpha=0.64, copy_X=True, fit_intercept=True, max_iter=None,
      normalize=False, random_state=None, solver='auto', tol=0.001)

Ridge(alpha=1.28, copy_X=True, fit_intercept=True, max_iter=None,
      normalize=False, random_state=None, solver='auto', tol=0.001)

Ridge(alpha=2.56, copy_X=True, fit_intercept=True, max_iter=None,
      normalize=False, random_state=None, solver='auto', tol=0.001)

Ridge(alpha=5.12, copy_X=True, fit_intercept=True, max_iter=None,
      normalize=False, random_state=None, solver='auto', tol=0.001)

alpha_results = pd.DataFrame(data = {'Alpha': alpha_vals, 'Training_Errors': train_errors, 'Validation_Errors': validation_errors})
alpha_results

fig = go.Figure(data=go.Scatter(
                x=alpha_results['Alpha'],
                y=alpha_results['Training_Errors'],
                mode='lines+markers',
                name = 'Training Error',
                hovertemplate='Alpha: %{x} <br>Log RMSE: %{y} '
));

fig.add_trace(go.Scatter(
                x=alpha_results['Alpha'],
                y=alpha_results['Validation_Errors'],
                mode='lines+markers',
                name = 'Validation Error',
                hovertemplate='Alpha: %{x} <br>Log RMSE: %{y} '
));

fig.update_layout(
    title='Impact of Regularisation Parameter on Log RMSE of Training and Validation Sets',
    xaxis_title='Regularisation Parameter, Alpha',
    yaxis_title='Log RMSE'
)

It is favourable to reduce the Validation Error, since this best emulates the Test Set. I now, apply a Ridge Model with alpha parameter set to 20 to predict house prices in my test set.

model = Ridge(alpha=20, fit_intercept=True)
model.fit(X_train, y_train) # Fit the model to the training data
print('Model Intercept:',round(model.intercept_,2))
print('Model Coefficients:', model.coef_)

Ridge(alpha=20, copy_X=True, fit_intercept=True, max_iter=None, normalize=False,
      random_state=None, solver='auto', tol=0.001)

Model Intercept: 10.44
Model Coefficients: [ 5.11783709e-04  4.05126770e-06  5.03915118e-02  4.59720532e-02
  2.14961870e-05  7.49667016e-05  4.20016860e-05 -3.32440026e-06
  1.13643988e-04  8.70777167e-05  8.89350560e-05 -3.87040153e-05
  1.37308759e-04  1.82232476e-02 -4.82936574e-03  2.81762060e-02
  2.86405668e-02 -7.74610088e-03 -4.41679412e-02  7.92422007e-03
  1.43416232e-02  2.74555392e-02  9.52209940e-05  1.04149758e-04
  8.57104979e-05  2.23958323e-05  2.03766620e-04  1.97260757e-04
  1.08352841e-04 -4.46591736e-06  0.00000000e+00 -2.22873800e-02
 -4.85582490e-03 -1.61255448e-02  1.25247118e-02 -5.30364192e-02
  8.86232369e-03 -1.67241675e-03 -3.45827817e-03  1.04643428e-02
  1.91051764e-02  5.47253301e-03  6.32760662e-03  8.61451974e-03
  7.97679221e-03  3.77629603e-02  2.71888281e-02  3.53124154e-02
 -9.30356120e-03  1.62214738e-02 -1.11769174e-02  1.70968349e-02
  1.38487124e-02  8.21159736e-04  3.13006963e-03  1.56431278e-02
 -6.73660636e-03  9.30690889e-03 -5.86327701e-03  2.82996798e-02
 -2.06076406e-02 -4.20765782e-03 -5.64903463e-03  1.41305318e-03
 -2.10936853e-02 -2.07600636e-03 -1.79162421e-03  2.36239513e-02
  1.38972746e-02 -4.18631731e-03  6.11567341e-02 -2.83221403e-02
 -2.93506663e-03 -3.83571404e-02 -2.92555753e-02 -1.09638247e-02
 -1.42427221e-02  6.88644753e-03 -1.77103499e-02  3.98456068e-03
  3.41166604e-02 -3.80258592e-02 -1.02513854e-02 -1.18512816e-02
  1.11362985e-03  2.70780910e-02  4.73415262e-02 -1.74276809e-03
  4.34462041e-03  1.41409260e-02  4.36223103e-02 -5.21925445e-03
  1.13206438e-02 -2.17147615e-02  5.29329106e-03  2.40206791e-03
  7.84080190e-03  6.16603222e-03  9.95040363e-03  5.69864132e-04
 -5.55940365e-03 -2.95191333e-03  0.00000000e+00  0.00000000e+00
 -1.36155204e-02  7.97679221e-03 -1.35838304e-02  7.89778324e-03
 -4.90121155e-03 -3.77028050e-03 -5.66870307e-03  4.29484770e-03
 -9.33785163e-03 -2.20708390e-03  1.69782889e-02  0.00000000e+00
  0.00000000e+00 -1.29913486e-03 -4.95360108e-03  5.80390604e-04
 -8.54656783e-03  1.97604901e-03  5.69864132e-04  0.00000000e+00
  0.00000000e+00 -6.15879668e-03 -2.06544750e-02  0.00000000e+00
  0.00000000e+00  9.64327044e-04  5.26416112e-03  5.49302903e-04
  0.00000000e+00 -2.68993999e-03 -2.20303489e-02  3.49006391e-03
  1.55791424e-03 -4.51097861e-03 -3.65913870e-03  1.65103007e-03
  9.83441986e-03  0.00000000e+00 -1.02040697e-02  1.96590179e-03
 -3.87571943e-02  1.11799986e-02 -7.22979687e-03  1.88311570e-02
 -1.37293841e-02 -8.24730747e-03 -1.15999880e-02 -1.10779877e-02
  9.52885242e-03  6.24494784e-03 -8.76753408e-03 -2.13994172e-03
  1.63813864e-02  9.78908982e-03  1.36051327e-03  1.17546564e-02
 -2.88464710e-03 -2.56837430e-03 -3.33682439e-03  3.79862375e-03
 -1.94539805e-02 -8.97743103e-05  5.12995787e-03 -6.06907722e-03
  1.13829278e-03  2.93841884e-03 -5.91353088e-03  5.07929590e-03
  7.65852196e-03 -1.66429790e-03  3.10367149e-04  1.07458247e-02
  7.06047889e-03 -1.26602660e-03 -3.19401619e-03  1.01634513e-02
 -2.53879357e-03  1.36706291e-02  9.20775770e-04  1.02677684e-02
 -8.78289856e-03 -1.17004869e-02 -1.43113201e-02 -1.71065397e-04
 -1.08880305e-03 -1.02060524e-02 -4.71114033e-03 -3.73516154e-03
  5.03952326e-03 -1.30804708e-02 -1.60920318e-02  1.07440814e-02
 -2.01494003e-03 -1.69333514e-02 -4.32078108e-03  1.10044499e-04
  5.67343941e-03  6.83618073e-03 -8.96530066e-03  7.77566133e-03
  9.24160455e-05 -6.00086569e-03  4.90709791e-03  5.36305270e-03
  1.64757657e-02 -6.80509736e-03  5.31680142e-03 -6.04652691e-03
 -1.27468204e-03 -8.70562040e-03 -5.63771305e-03 -2.94744263e-03
  1.21676776e-02  1.22766940e-02  5.40027361e-03  7.71167467e-03
  3.42068153e-03  2.16863472e-03  3.78805141e-03 -4.77095728e-03
 -3.76739034e-03 -1.69155002e-03  1.59676894e-02 -4.26608354e-03
  2.32493944e-02  3.71813515e-02  1.65397978e-02  9.63723777e-03
  6.02020330e-03 -1.66429790e-03 -5.56513416e-03  4.27235175e-04
  1.70334645e-02 -8.45206053e-03  1.14441090e-03 -6.44523115e-03
 -6.67977186e-03  3.60284025e-03  6.02410360e-03  2.07190410e-02
 -1.12335486e-02 -5.60996800e-04 -1.52062828e-02  1.10523658e-02
  5.69678691e-06  6.43581595e-03 -9.18104470e-03  2.98466176e-03
  3.70526534e-03 -8.61132090e-03 -1.22462221e-02  4.13610439e-03
 -8.24292331e-03  3.74725571e-04 -4.83489850e-03 -1.86225905e-03
 -1.86612190e-04  4.73607444e-04 -9.00297902e-03  1.04227151e-02
  3.28506213e-03  4.00478159e-03  1.92748830e-04  1.66020511e-03
  1.07941273e-02  9.48541345e-03 -6.82107176e-03 -8.64618696e-03
 -3.39718266e-03 -2.53186405e-04  4.88510174e-03  1.31645183e-02
  8.29367447e-03  5.23941904e-03  1.21293341e-02  1.98541495e-03
  5.01384032e-03  1.55037926e-02  1.81609022e-02  6.67033501e-03
 -4.94412850e-03 -9.75137478e-04 -1.28620538e-03 -7.17025370e-03
 -7.51933968e-03  1.88747794e-02  2.76707830e-02  1.19503972e-02
 -7.74593352e-03  1.06881120e-02 -2.70773242e-03  1.55524680e-03
  6.37607058e-04  0.00000000e+00  3.97176198e-03 -2.65570104e-03
 -8.23933441e-03  3.85236383e-03  2.11224708e-03  0.00000000e+00
 -1.53884372e-02  4.13328558e-02 -3.54142347e-03  4.25939980e-03
 -1.14004225e-03  0.00000000e+00  6.76516449e-03  2.00940221e-04
  4.14858955e-03 -1.70378259e-03  1.89174706e-02 -3.24636952e-02
  7.98355681e-03  5.33500470e-03 -5.61745779e-03  4.31376605e-03
 -3.54142347e-03  1.02295332e-02 -3.46589572e-04  6.58552480e-03
  2.46801953e-03 -1.70333315e-03  2.87807429e-03 -9.03632567e-04
 -2.24068038e-03  4.30810496e-03  1.14400976e-02 -1.04156581e-02
  5.98139695e-04 -4.15040627e-04  1.69111115e-02 -1.01126809e-02
 -5.36765283e-03 -2.15393020e-02  6.59328736e-03 -1.39483101e-02
  1.41545315e-03  3.70515171e-03 -3.81066071e-03  3.93492339e-02
  3.85254190e-03 -4.04442143e-03 -1.04053479e-02 -1.59885333e-02
 -1.58140865e-02  1.94237560e-03 -3.86244105e-02  1.32789506e-02
  1.94237560e-03  1.27981759e-03  2.00546438e-02  3.02126060e-02
  1.12747713e-03 -4.95100427e-03  1.94237560e-03  4.71508204e-03
  9.27844051e-03 -7.11112923e-03  1.94237560e-03 -1.92003496e-02
 -1.31841947e-02 -2.09387286e-02  9.74613042e-03 -7.04984038e-04
 -2.79484534e-03 -3.57584779e-03  1.36303445e-02  2.15376392e-02
  1.08785192e-03 -8.75231055e-03 -6.93639356e-03 -6.57453064e-04
 -8.49111619e-03 -2.21462155e-02 -1.93242596e-03 -2.50380851e-02
  6.00437979e-02  6.97211603e-03 -2.47984815e-03 -1.87655506e-03
  1.02983574e-03 -1.90711336e-02 -1.95413798e-02 -2.71264573e-02
 -1.37725356e-02  3.09788052e-03  1.33218465e-02 -3.23474647e-02
  0.00000000e+00  5.31170352e-02 -6.47397284e-03  6.84453885e-03
 -8.52361298e-03  6.20113096e-05 -6.48634935e-03  2.08524503e-02
 -1.54114045e-02  2.06124572e-02 -1.23396554e-02  2.93466970e-03
 -2.91536385e-03  0.00000000e+00 -9.09415049e-03  5.49302903e-04
 -2.68993999e-03 -8.31154557e-04  1.23840350e-03 -6.26635225e-03
 -7.16365632e-03  0.00000000e+00 -8.12398706e-03  0.00000000e+00
 -2.04349777e-03  9.32851051e-03  1.30644367e-02  1.04245325e-02
 -1.24278170e-02 -1.15740193e-02  2.30295991e-03  6.10627862e-03
 -9.49829360e-03  1.84287967e-03  7.96973320e-03  3.87798232e-03
  6.45585070e-03  1.25834442e-03  6.32667610e-03  9.60262293e-03
  7.25529023e-03  4.90694440e-03 -1.30204581e-02  1.24051260e-03
  1.22058496e-02 -1.88383823e-02 -8.97743103e-05  0.00000000e+00
 -1.00935841e-02 -3.32626138e-03  1.52670639e-03  2.05756445e-03
  3.84974819e-03  2.50511892e-03  8.15197357e-03 -1.90204329e-02
 -1.25476447e-02  3.58912604e-03 -8.12187760e-03  9.00352049e-04
 -6.46303319e-03 -2.22986911e-02 -1.51498627e-04  7.74824582e-03
 -1.56037072e-02  9.87333608e-03 -6.41613364e-03  8.79544821e-03
 -1.38100733e-02  4.33709178e-03 -1.31523893e-02  6.29499301e-03
 -8.23505092e-03 -4.29893284e-03 -3.67593550e-03  4.55077344e-03
 -1.60920318e-02  1.98521353e-02 -8.46608076e-03 -1.11124231e-02
 -2.39485750e-02 -1.29700045e-02  1.66095458e-02 -5.66962348e-03
 -5.82031800e-04 -5.33314641e-03  2.95124749e-03 -6.91120950e-03
 -3.64845170e-03  4.92152154e-03  1.02099963e-02 -1.64410289e-03
  5.47232468e-03 -1.71744807e-02  1.36439397e-02 -1.16021402e-02
  8.16649198e-04 -3.29756457e-03  5.40195578e-03 -1.08103988e-03
  1.54819049e-02  3.86683214e-03  4.48921595e-03  1.62985426e-02
  3.78805141e-03 -5.73406094e-03  7.54714399e-03  3.36200497e-03
  6.56519480e-04  2.10570909e-02 -2.92166161e-03  1.96767892e-02
  3.04879065e-02  1.03364515e-02 -2.91536385e-03 -6.01557620e-03
 -1.14136138e-02 -1.35061545e-02  1.01438991e-02 -2.91536385e-03
 -2.76520619e-03 -3.59509087e-03 -7.58939167e-03  7.68972760e-04
 -2.91536385e-03 -3.21333185e-03  8.58726077e-03 -8.78919025e-03
  3.88753481e-02 -8.73187136e-03  1.30745982e-02  3.96316488e-03
 -4.76887427e-03  5.54877547e-03 -1.77573859e-03  6.69379078e-03
  1.42709533e-02 -7.50032958e-03  6.35248285e-04 -8.49943264e-03
 -5.19252660e-03 -1.22286322e-02  6.39258340e-03 -2.06318465e-03
  1.89925161e-03 -1.82232400e-03  1.35336067e-02  3.04191975e-03
  9.96180925e-03  9.99255458e-03 -7.15105533e-03  1.32582332e-03
 -6.39315307e-03 -1.92498015e-02 -1.00313979e-02  7.53602693e-03
  1.86443723e-03  1.42867345e-02 -8.88347625e-03 -4.98687787e-03
  4.23295806e-02  1.50518749e-02 -2.57635244e-02 -1.34934221e-03
 -3.47651781e-04 -7.86265025e-03  5.31090704e-02  2.53096886e-02]

Result

An Improvement of a 0.13298 RMSE to a 0.12540 RMSE. This pushed me up over 500 places, into position 1096.

Model 5: Lasso Regularised Regression¶

Lasso uses L1 penalties (Sum of the coefficients)

alpha_vals = [0.0001,0.00025,0.0005,0.0007,0.001,0.0025,0.005,0.01,0.02]
train_errors = []
validation_errors = []

for a in alpha_vals:
    model = Lasso(alpha=a, fit_intercept=True);
    model.fit(X_train, y_train); # Fit the model to the training data
    
    train_results = pd.DataFrame(y_train)
    train_results['Model_Output(Log)'] = model.predict(X_train)
    train_errors.append(np.sqrt(metrics.mean_squared_error(train_results['LogSalePrice'], train_results['Model_Output(Log)'])))
    #px.scatter(train_results,x='Model_Output(Log)',y='LogSalePrice',title='Train Output: Alpha = {}'.format(a))

    val_results = pd.DataFrame(y_val)
    val_results['Model_Predictions(Log)'] = model.predict(X_val)
    validation_errors.append(np.sqrt(metrics.mean_squared_error(val_results['LogSalePrice'], val_results['Model_Predictions(Log)'])))
    #px.scatter(val_results,x='Model_Predictions(Log)',y='LogSalePrice',title='Validation Output: Alpha = {}'.format(a))

Lasso(alpha=0.0001, copy_X=True, fit_intercept=True, max_iter=1000,
      normalize=False, positive=False, precompute=False, random_state=None,
      selection='cyclic', tol=0.0001, warm_start=False)

Lasso(alpha=0.00025, copy_X=True, fit_intercept=True, max_iter=1000,
      normalize=False, positive=False, precompute=False, random_state=None,
      selection='cyclic', tol=0.0001, warm_start=False)

Lasso(alpha=0.0005, copy_X=True, fit_intercept=True, max_iter=1000,
      normalize=False, positive=False, precompute=False, random_state=None,
      selection='cyclic', tol=0.0001, warm_start=False)

Lasso(alpha=0.0007, copy_X=True, fit_intercept=True, max_iter=1000,
      normalize=False, positive=False, precompute=False, random_state=None,
      selection='cyclic', tol=0.0001, warm_start=False)

Lasso(alpha=0.001, copy_X=True, fit_intercept=True, max_iter=1000,
      normalize=False, positive=False, precompute=False, random_state=None,
      selection='cyclic', tol=0.0001, warm_start=False)

Lasso(alpha=0.0025, copy_X=True, fit_intercept=True, max_iter=1000,
      normalize=False, positive=False, precompute=False, random_state=None,
      selection='cyclic', tol=0.0001, warm_start=False)

Lasso(alpha=0.005, copy_X=True, fit_intercept=True, max_iter=1000,
      normalize=False, positive=False, precompute=False, random_state=None,
      selection='cyclic', tol=0.0001, warm_start=False)

Lasso(alpha=0.01, copy_X=True, fit_intercept=True, max_iter=1000,
      normalize=False, positive=False, precompute=False, random_state=None,
      selection='cyclic', tol=0.0001, warm_start=False)

Lasso(alpha=0.02, copy_X=True, fit_intercept=True, max_iter=1000,
      normalize=False, positive=False, precompute=False, random_state=None,
      selection='cyclic', tol=0.0001, warm_start=False)

alpha_results = pd.DataFrame(data = {'Alpha': alpha_vals, 'Training_Errors': train_errors, 'Validation_Errors': validation_errors})
alpha_results

fig = go.Figure(data=go.Scatter(
                x=alpha_results['Alpha'],
                y=alpha_results['Training_Errors'],
                mode='lines+markers',
                name = 'Training Error',
                hovertemplate='Alpha: %{x} <br>Log RMSE: %{y} '
));

fig.add_trace(go.Scatter(
                x=alpha_results['Alpha'],
                y=alpha_results['Validation_Errors'],
                mode='lines+markers',
                name = 'Validation Error',
                hovertemplate='Alpha: %{x} <br>Log RMSE: %{y} '
));

fig.update_layout(
    title='Impact of Regularisation Parameter on Log RMSE of Training and Validation Sets',
    xaxis_title='Regularisation Parameter, Alpha',
    yaxis_title='Log RMSE'
)

We see that the validation set error is minimised with alpha set to around 0.0005. I apply this model below and submit.

model = Lasso(alpha=0.0005, fit_intercept=True)
model.fit(X_train, y_train) # Fit the model to the training data
print('Model Intercept:',round(model.intercept_,2))
print('Model Coefficients:', model.coef_)

Lasso(alpha=0.0005, copy_X=True, fit_intercept=True, max_iter=1000,
      normalize=False, positive=False, precompute=False, random_state=None,
      selection='cyclic', tol=0.0001, warm_start=False)

Model Intercept: 10.43
Model Coefficients: [ 5.26930614e-04  4.12858108e-06  5.08305982e-02  4.51149463e-02
  3.14599619e-05  1.38559752e-04  1.02087831e-04  5.80112838e-05
  3.85175493e-05  2.12816795e-04  1.92705587e-04  6.72155313e-05
  2.97032880e-05  1.86357213e-02 -0.00000000e+00  2.86704855e-02
  2.74007487e-02 -7.38834611e-03 -6.76054410e-02  7.70657391e-03
  1.32254757e-02  3.13284854e-02  8.89037998e-05  1.07953034e-04
  1.12087354e-04  7.05058597e-06  1.66125054e-04  1.72830148e-04
  1.20688753e-04 -3.12736891e-06  0.00000000e+00 -3.16983219e-02
 -0.00000000e+00 -1.60006471e-02 -0.00000000e+00 -7.60918233e-02
  0.00000000e+00 -0.00000000e+00 -2.38145038e-03  0.00000000e+00
  0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00
  0.00000000e+00  6.10599712e-02  4.86259035e-02  4.56367278e-02
  0.00000000e+00  0.00000000e+00 -1.14309805e-05  1.80608884e-02
  4.31703354e-03  0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00  3.81162068e-03 -0.00000000e+00  3.14443299e-02
 -7.28441011e-03 -0.00000000e+00 -1.80347537e-03 -0.00000000e+00
 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00  4.40175793e-02
  7.24343583e-03 -0.00000000e+00  9.30426640e-02 -2.10505782e-02
  0.00000000e+00 -2.43242563e-02 -3.35933653e-02 -0.00000000e+00
 -0.00000000e+00  0.00000000e+00 -1.20401858e-02  0.00000000e+00
  4.66675213e-02 -2.40686782e-02 -0.00000000e+00 -0.00000000e+00
  0.00000000e+00  3.19962973e-02  8.42431166e-02  0.00000000e+00
  0.00000000e+00  5.73478546e-03  4.77182400e-02 -0.00000000e+00
  0.00000000e+00 -6.32824993e-03  0.00000000e+00  0.00000000e+00
  0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00  0.00000000e+00  0.00000000e+00
 -0.00000000e+00  0.00000000e+00 -1.21959536e-02  0.00000000e+00
 -0.00000000e+00  0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00  0.00000000e+00  3.99819035e-03  0.00000000e+00
  0.00000000e+00 -0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00
  0.00000000e+00 -0.00000000e+00 -0.00000000e+00  0.00000000e+00
  0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00
  0.00000000e+00 -0.00000000e+00 -2.06270502e-02  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00  0.00000000e+00
  0.00000000e+00  0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -5.10584531e-02  0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00
  0.00000000e+00  0.00000000e+00 -0.00000000e+00 -0.00000000e+00
  0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -1.17762994e-02  0.00000000e+00  0.00000000e+00 -0.00000000e+00
  0.00000000e+00  0.00000000e+00 -0.00000000e+00  0.00000000e+00
  0.00000000e+00 -0.00000000e+00 -0.00000000e+00  0.00000000e+00
  0.00000000e+00 -0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00  0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00 -6.30022613e-03  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00
  0.00000000e+00 -0.00000000e+00 -1.82769216e-04  0.00000000e+00
 -0.00000000e+00 -7.21237306e-03 -0.00000000e+00 -0.00000000e+00
  0.00000000e+00  0.00000000e+00 -0.00000000e+00  0.00000000e+00
  0.00000000e+00 -0.00000000e+00  0.00000000e+00  0.00000000e+00
  0.00000000e+00 -0.00000000e+00  0.00000000e+00 -0.00000000e+00
  0.00000000e+00 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00
  0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00
  0.00000000e+00  0.00000000e+00  0.00000000e+00 -0.00000000e+00
 -0.00000000e+00 -0.00000000e+00  2.63746663e-03 -0.00000000e+00
  2.00570189e-02  6.41656252e-02  0.00000000e+00  0.00000000e+00
  0.00000000e+00 -0.00000000e+00 -0.00000000e+00  0.00000000e+00
  0.00000000e+00 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00
 -0.00000000e+00  0.00000000e+00  0.00000000e+00  2.45224808e-02
 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00  0.00000000e+00 -0.00000000e+00 -0.00000000e+00
  0.00000000e+00 -0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00
  0.00000000e+00  0.00000000e+00 -0.00000000e+00  0.00000000e+00
  0.00000000e+00  0.00000000e+00 -0.00000000e+00  0.00000000e+00
  0.00000000e+00  0.00000000e+00 -0.00000000e+00 -0.00000000e+00
 -0.00000000e+00 -0.00000000e+00  0.00000000e+00  0.00000000e+00
  0.00000000e+00  0.00000000e+00  2.56287153e-03  0.00000000e+00
  0.00000000e+00  1.30765084e-02  5.14917033e-03  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00  0.00000000e+00 -0.00000000e+00
 -2.71204495e-03  7.64048592e-03  1.32336155e-02  0.00000000e+00
 -2.08061427e-03  0.00000000e+00  0.00000000e+00  0.00000000e+00
  0.00000000e+00  0.00000000e+00  0.00000000e+00 -0.00000000e+00
 -0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00
 -0.00000000e+00  6.08134356e-02 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00  0.00000000e+00  6.99660445e-03 -0.00000000e+00
  0.00000000e+00 -0.00000000e+00  2.62187392e-02 -2.25288955e-02
  0.00000000e+00  0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00  0.00000000e+00 -9.94190473e-04  0.00000000e+00
  0.00000000e+00 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00
 -0.00000000e+00  0.00000000e+00  0.00000000e+00 -8.88868614e-04
 -0.00000000e+00 -0.00000000e+00  1.10802538e-02 -0.00000000e+00
  0.00000000e+00 -1.50391637e-02  0.00000000e+00 -1.39981134e-02
  0.00000000e+00  0.00000000e+00  0.00000000e+00  4.82475934e-02
  0.00000000e+00 -0.00000000e+00 -0.00000000e+00 -8.57965481e-03
 -1.15242949e-02  0.00000000e+00 -3.92592828e-02  0.00000000e+00
  0.00000000e+00 -0.00000000e+00  9.83063713e-03  3.77388950e-02
  0.00000000e+00 -0.00000000e+00  0.00000000e+00  0.00000000e+00
  8.61978095e-03 -0.00000000e+00  0.00000000e+00 -1.75290129e-02
 -6.90377672e-03 -1.83333690e-02  0.00000000e+00 -0.00000000e+00
 -0.00000000e+00 -0.00000000e+00  8.04740723e-03  1.03513505e-02
 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00
 -2.34373027e-03 -2.24246541e-02 -0.00000000e+00 -2.58144361e-02
  8.12952238e-02  0.00000000e+00  0.00000000e+00 -0.00000000e+00
 -0.00000000e+00 -9.18794224e-03 -1.87850881e-02 -2.58078706e-02
 -0.00000000e+00  0.00000000e+00  7.94869226e-03 -5.03985849e-02
  0.00000000e+00  6.30280238e-02 -0.00000000e+00  6.41637116e-03
 -3.86995838e-03 -0.00000000e+00 -0.00000000e+00  1.59096256e-02
 -3.64160678e-03  1.67424802e-02 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00  0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00
 -0.00000000e+00  0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00  0.00000000e+00  0.00000000e+00
  0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00
  0.00000000e+00  0.00000000e+00 -0.00000000e+00  0.00000000e+00
  0.00000000e+00 -1.73323009e-03  0.00000000e+00  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00  0.00000000e+00  0.00000000e+00
  0.00000000e+00  0.00000000e+00  0.00000000e+00 -0.00000000e+00
 -0.00000000e+00  0.00000000e+00 -0.00000000e+00 -0.00000000e+00
 -0.00000000e+00 -2.81391711e-03 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00  0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -1.50216497e-02  0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00  0.00000000e+00  0.00000000e+00
 -1.26034441e-04  1.32100683e-02 -0.00000000e+00 -0.00000000e+00
 -2.23562908e-02 -0.00000000e+00  2.27541687e-03 -0.00000000e+00
 -0.00000000e+00 -0.00000000e+00  0.00000000e+00 -0.00000000e+00
 -0.00000000e+00  0.00000000e+00  0.00000000e+00 -0.00000000e+00
  0.00000000e+00 -0.00000000e+00  0.00000000e+00 -0.00000000e+00
 -0.00000000e+00 -0.00000000e+00  0.00000000e+00  0.00000000e+00
  6.42357299e-03  0.00000000e+00  0.00000000e+00  0.00000000e+00
  0.00000000e+00 -0.00000000e+00  0.00000000e+00 -0.00000000e+00
  0.00000000e+00  3.47114370e-03 -0.00000000e+00  3.67204319e-03
  4.83804360e-02  0.00000000e+00 -0.00000000e+00 -1.59466805e-03
 -3.37633886e-03 -5.86075732e-03  0.00000000e+00 -0.00000000e+00
 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00  0.00000000e+00 -0.00000000e+00
  4.03875935e-02 -0.00000000e+00  0.00000000e+00  0.00000000e+00
 -0.00000000e+00  0.00000000e+00 -0.00000000e+00  2.21104476e-03
  7.89769946e-03 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00
 -1.01669137e-03 -6.77040533e-03  0.00000000e+00 -0.00000000e+00
 -0.00000000e+00 -0.00000000e+00  8.82178342e-03  0.00000000e+00
  7.19799137e-03  4.07046653e-03 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00 -1.48296789e-02 -3.52924818e-03  0.00000000e+00
  0.00000000e+00  0.00000000e+00 -0.00000000e+00 -0.00000000e+00
  8.73118720e-02  0.00000000e+00 -2.33513558e-02  0.00000000e+00
  0.00000000e+00 -0.00000000e+00  6.26754553e-02  0.00000000e+00]

Result

An Improvement of a 0.12540 RMSE to a 0.12502 RMSE. This pushed me up 28 places, into position 1051.

Model 6: Elastic Net Regularised Regression¶

Elastic Net takes a combination of L1 and L2 penalties. It has hyperparameters, alpha (as before) and

alpha_vals = [0.0001,0.00025,0.0005,0.0007,0.001,0.0025,0.005,0.01,0.02,0.04,0.1]
l1_ratios = [0.1,0.2,0.3,0.4,0.5,0.6,0.7,0.8,0.9]
train_errors = []
validation_errors = []
alpha_order = []
ratio_order = []

for a in alpha_vals:
        for r in l1_ratios:
            alpha_order.append(a)
            ratio_order.append(r)
            model =ElasticNet(alpha=a, l1_ratio=r, fit_intercept=True, normalize=False)
            model.fit(X_train, y_train); # Fit the model to the training data

            train_results = pd.DataFrame(y_train)
            train_results['Model_Output(Log)'] = model.predict(X_train)
            train_errors.append(np.sqrt(metrics.mean_squared_error(train_results['LogSalePrice'], train_results['Model_Output(Log)'])))
            #px.scatter(train_results,x='Model_Output(Log)',y='LogSalePrice',title='Train Output: Alpha = {}'.format(a))

            val_results = pd.DataFrame(y_val)
            val_results['Model_Predictions(Log)'] = model.predict(X_val)
            validation_errors.append(np.sqrt(metrics.mean_squared_error(val_results['LogSalePrice'], val_results['Model_Predictions(Log)'])))
            #px.scatter(val_results,x='Model_Predictions(Log)',y='LogSalePrice',title='Validation Output: Alpha = {}'.format(a))

ElasticNet(alpha=0.0001, copy_X=True, fit_intercept=True, l1_ratio=0.1,
           max_iter=1000, normalize=False, positive=False, precompute=False,
           random_state=None, selection='cyclic', tol=0.0001, warm_start=False)

ElasticNet(alpha=0.0001, copy_X=True, fit_intercept=True, l1_ratio=0.2,
           max_iter=1000, normalize=False, positive=False, precompute=False,
           random_state=None, selection='cyclic', tol=0.0001, warm_start=False)

ElasticNet(alpha=0.0001, copy_X=True, fit_intercept=True, l1_ratio=0.3,
           max_iter=1000, normalize=False, positive=False, precompute=False,
           random_state=None, selection='cyclic', tol=0.0001, warm_start=False)

ElasticNet(alpha=0.0001, copy_X=True, fit_intercept=True, l1_ratio=0.4,
           max_iter=1000, normalize=False, positive=False, precompute=False,
           random_state=None, selection='cyclic', tol=0.0001, warm_start=False)

ElasticNet(alpha=0.0001, copy_X=True, fit_intercept=True, l1_ratio=0.5,
           max_iter=1000, normalize=False, positive=False, precompute=False,
           random_state=None, selection='cyclic', tol=0.0001, warm_start=False)

ElasticNet(alpha=0.0001, copy_X=True, fit_intercept=True, l1_ratio=0.6,
           max_iter=1000, normalize=False, positive=False, precompute=False,
           random_state=None, selection='cyclic', tol=0.0001, warm_start=False)

ElasticNet(alpha=0.0001, copy_X=True, fit_intercept=True, l1_ratio=0.7,
           max_iter=1000, normalize=False, positive=False, precompute=False,
           random_state=None, selection='cyclic', tol=0.0001, warm_start=False)

ElasticNet(alpha=0.0001, copy_X=True, fit_intercept=True, l1_ratio=0.8,
           max_iter=1000, normalize=False, positive=False, precompute=False,
           random_state=None, selection='cyclic', tol=0.0001, warm_start=False)

ElasticNet(alpha=0.0001, copy_X=True, fit_intercept=True, l1_ratio=0.9,
           max_iter=1000, normalize=False, positive=False, precompute=False,
           random_state=None, selection='cyclic', tol=0.0001, warm_start=False)

ElasticNet(alpha=0.00025, copy_X=True, fit_intercept=True, l1_ratio=0.1,
           max_iter=1000, normalize=False, positive=False, precompute=False,
           random_state=None, selection='cyclic', tol=0.0001, warm_start=False)

alpha_results = pd.DataFrame(data = {'Alpha': alpha_order, 'Ratio': ratio_order, 'Training_Errors': train_errors, 'Validation_Errors': validation_errors})
alpha_results.sort_values('Validation_Errors')

We see that the validation set error is minimised with alpha set to around 0.005. I apply this model below and submit.

model = ElasticNet(alpha=0.005,l1_ratio = 0.1, fit_intercept=True)
model.fit(X_train, y_train) # Fit the model to the training data
print('Model Intercept:',round(model.intercept_,2))
print('Model Coefficients:', model.coef_)

ElasticNet(alpha=0.005, copy_X=True, fit_intercept=True, l1_ratio=0.1,
           max_iter=1000, normalize=False, positive=False, precompute=False,
           random_state=None, selection='cyclic', tol=0.0001, warm_start=False)

Model Intercept: 10.44
Model Coefficients: [ 5.44812295e-04  4.15483358e-06  5.28208656e-02  4.54359226e-02
  3.10593134e-05  1.41140336e-04  1.02434725e-04  5.89001872e-05
  3.90207851e-05  2.12305214e-04  1.92555420e-04  6.63013783e-05
  2.96472976e-05  1.81470441e-02 -0.00000000e+00  2.63164907e-02
  2.70991956e-02 -8.43277237e-03 -5.82311487e-02  7.97980069e-03
  1.36540336e-02  2.92362760e-02  9.56005490e-05  1.04159106e-04
  1.10850386e-04  7.89672809e-06  1.75647259e-04  1.75032892e-04
  1.11609256e-04 -3.75364443e-06  0.00000000e+00 -2.62741267e-02
 -0.00000000e+00 -1.37417512e-02  0.00000000e+00 -6.58312604e-02
  0.00000000e+00 -0.00000000e+00 -0.00000000e+00  0.00000000e+00
  3.55712153e-03  0.00000000e+00  0.00000000e+00  0.00000000e+00
  0.00000000e+00  4.79914369e-02  2.60086435e-02  4.12758719e-02
 -0.00000000e+00  0.00000000e+00 -6.29835324e-05  1.83425083e-02
  3.86284016e-03  0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00  8.61873807e-04 -0.00000000e+00  2.93901705e-02
 -6.47356849e-03 -0.00000000e+00 -1.63916200e-03 -0.00000000e+00
 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00  3.10385784e-02
  3.54154089e-03 -0.00000000e+00  7.55698558e-02 -2.22567595e-02
  0.00000000e+00 -3.30805517e-02 -2.32646940e-02 -0.00000000e+00
 -1.09709400e-03  0.00000000e+00 -1.20181715e-02  0.00000000e+00
  3.96211967e-02 -3.40916828e-02 -0.00000000e+00 -0.00000000e+00
  0.00000000e+00  3.22796862e-02  6.23603748e-02 -0.00000000e+00
  0.00000000e+00  3.57499184e-03  4.56192482e-02 -0.00000000e+00
  0.00000000e+00 -5.22703644e-03  0.00000000e+00  0.00000000e+00
  0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00  0.00000000e+00  0.00000000e+00
 -5.65050619e-04  0.00000000e+00 -1.35422025e-02  0.00000000e+00
 -0.00000000e+00  0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00  0.00000000e+00  3.26570930e-03  0.00000000e+00
  0.00000000e+00 -0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00
  0.00000000e+00 -0.00000000e+00 -0.00000000e+00  0.00000000e+00
  0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00
  0.00000000e+00 -0.00000000e+00 -1.33782576e-02  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00  0.00000000e+00
  0.00000000e+00  0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -4.40455983e-02  0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00
  0.00000000e+00  0.00000000e+00 -0.00000000e+00 -0.00000000e+00
  0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -7.07049021e-03  0.00000000e+00  0.00000000e+00 -0.00000000e+00
  0.00000000e+00  0.00000000e+00 -0.00000000e+00  0.00000000e+00
  0.00000000e+00 -0.00000000e+00 -0.00000000e+00  0.00000000e+00
  0.00000000e+00 -0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00  0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00 -5.52576763e-03  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00
  0.00000000e+00 -0.00000000e+00 -2.77512181e-03  0.00000000e+00
 -0.00000000e+00 -8.08684546e-03 -0.00000000e+00 -0.00000000e+00
  0.00000000e+00  0.00000000e+00 -0.00000000e+00  0.00000000e+00
  0.00000000e+00 -0.00000000e+00  0.00000000e+00  0.00000000e+00
  0.00000000e+00 -0.00000000e+00  0.00000000e+00 -0.00000000e+00
  0.00000000e+00 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00
  0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00
  0.00000000e+00  0.00000000e+00  0.00000000e+00 -0.00000000e+00
 -0.00000000e+00 -0.00000000e+00  4.07017532e-03 -0.00000000e+00
  1.27012012e-02  4.14897671e-02  0.00000000e+00  0.00000000e+00
  0.00000000e+00 -0.00000000e+00 -0.00000000e+00  0.00000000e+00
  0.00000000e+00 -0.00000000e+00  0.00000000e+00 -0.00000000e+00
 -0.00000000e+00  0.00000000e+00  0.00000000e+00  1.60499816e-02
 -0.00000000e+00 -0.00000000e+00 -8.45683072e-04  0.00000000e+00
 -0.00000000e+00  0.00000000e+00 -0.00000000e+00 -0.00000000e+00
  0.00000000e+00 -0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00
  0.00000000e+00  0.00000000e+00 -0.00000000e+00  0.00000000e+00
  0.00000000e+00  0.00000000e+00 -0.00000000e+00  0.00000000e+00
  0.00000000e+00  0.00000000e+00 -0.00000000e+00 -0.00000000e+00
 -0.00000000e+00 -0.00000000e+00  0.00000000e+00  0.00000000e+00
  0.00000000e+00  0.00000000e+00  4.64435932e-03  0.00000000e+00
  0.00000000e+00  1.19369371e-02  5.00304678e-03  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00  0.00000000e+00 -0.00000000e+00
 -3.33580289e-04  1.16153561e-02  2.42640656e-02  0.00000000e+00
 -3.03886206e-03  0.00000000e+00  0.00000000e+00  0.00000000e+00
  0.00000000e+00  0.00000000e+00  0.00000000e+00 -0.00000000e+00
 -0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00
 -0.00000000e+00  5.16584287e-02 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00  0.00000000e+00  6.26685028e-03 -0.00000000e+00
  0.00000000e+00 -0.00000000e+00  2.38469087e-02 -2.42963355e-02
  0.00000000e+00  0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00  0.00000000e+00 -1.59038852e-03  0.00000000e+00
  0.00000000e+00 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00
 -0.00000000e+00  0.00000000e+00  0.00000000e+00 -3.02387017e-03
 -0.00000000e+00  0.00000000e+00  1.23027191e-02 -0.00000000e+00
 -0.00000000e+00 -1.62347911e-02  0.00000000e+00 -1.34570535e-02
  0.00000000e+00  0.00000000e+00 -0.00000000e+00  4.93591594e-02
  0.00000000e+00 -0.00000000e+00 -0.00000000e+00 -1.01096562e-02
 -1.01707393e-02  0.00000000e+00 -3.80269874e-02  0.00000000e+00
  0.00000000e+00 -0.00000000e+00  9.50921277e-03  3.34129931e-02
 -0.00000000e+00 -1.01031060e-03  0.00000000e+00  0.00000000e+00
  8.71422322e-03 -0.00000000e+00  0.00000000e+00 -1.69799371e-02
 -5.98800755e-03 -1.59185885e-02  0.00000000e+00 -0.00000000e+00
  0.00000000e+00 -0.00000000e+00  7.49573361e-03  7.75497523e-03
 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00
 -2.00737182e-03 -2.03470300e-02 -0.00000000e+00 -2.54658326e-02
  7.51613975e-02  0.00000000e+00 -0.00000000e+00 -0.00000000e+00
 -0.00000000e+00 -1.08352458e-02 -1.60711422e-02 -2.36843026e-02
 -0.00000000e+00  0.00000000e+00  3.48752432e-03 -3.65713930e-02
  0.00000000e+00  5.91063906e-02 -0.00000000e+00  8.34421088e-03
 -3.90180081e-03 -0.00000000e+00 -0.00000000e+00  1.56267740e-02
 -4.66363173e-03  1.67981722e-02 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00  0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00  0.00000000e+00 -0.00000000e+00
 -0.00000000e+00  0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00  0.00000000e+00  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00  0.00000000e+00  0.00000000e+00
  0.00000000e+00  0.00000000e+00  0.00000000e+00  0.00000000e+00
  0.00000000e+00  0.00000000e+00 -0.00000000e+00  0.00000000e+00
  0.00000000e+00 -3.37358009e-03  0.00000000e+00  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00  0.00000000e+00  0.00000000e+00
  0.00000000e+00  0.00000000e+00  0.00000000e+00 -0.00000000e+00
 -0.00000000e+00  0.00000000e+00 -0.00000000e+00 -0.00000000e+00
 -0.00000000e+00 -3.56332058e-03 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00  0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -1.13183547e-02  0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00  0.00000000e+00  0.00000000e+00
 -2.77514559e-03  1.10861208e-02 -0.00000000e+00 -0.00000000e+00
 -1.99910024e-02 -0.00000000e+00  1.67578108e-03 -0.00000000e+00
 -0.00000000e+00 -0.00000000e+00  0.00000000e+00 -0.00000000e+00
 -0.00000000e+00  0.00000000e+00  0.00000000e+00 -0.00000000e+00
  0.00000000e+00 -0.00000000e+00  0.00000000e+00 -0.00000000e+00
 -0.00000000e+00 -0.00000000e+00  0.00000000e+00  0.00000000e+00
  3.71393303e-03  0.00000000e+00  0.00000000e+00  1.13480388e-03
  0.00000000e+00 -0.00000000e+00  0.00000000e+00 -0.00000000e+00
  0.00000000e+00  1.75093833e-03 -0.00000000e+00  1.11564087e-02
  3.53369210e-02  0.00000000e+00 -0.00000000e+00 -1.80435437e-03
 -4.95166941e-03 -3.24623403e-03  0.00000000e+00 -0.00000000e+00
 -0.00000000e+00  0.00000000e+00 -0.00000000e+00  0.00000000e+00
 -0.00000000e+00 -0.00000000e+00  3.24137065e-03 -0.00000000e+00
  3.92765156e-02 -0.00000000e+00  0.00000000e+00  0.00000000e+00
 -0.00000000e+00  0.00000000e+00 -0.00000000e+00  2.53464808e-03
  8.44285591e-03 -0.00000000e+00 -0.00000000e+00 -0.00000000e+00
 -1.87449800e-04 -7.36078098e-03  0.00000000e+00 -0.00000000e+00
 -0.00000000e+00 -0.00000000e+00  8.12678493e-03  0.00000000e+00
  7.80817257e-03  5.40792172e-03 -0.00000000e+00  4.14035195e-04
 -0.00000000e+00 -1.41166271e-02 -2.52480687e-03  0.00000000e+00
  0.00000000e+00  0.00000000e+00 -0.00000000e+00 -0.00000000e+00
  6.97734318e-02  0.00000000e+00 -2.36053121e-02  0.00000000e+00
  0.00000000e+00 -0.00000000e+00  5.61507488e-02  9.61649707e-03]

Result

No Improvement: This had an RMSE of 0.12523.

Submission Export ¶

This code takes the most recently defined model and outputs the predictions to csv.

# Reset the test data frame, and make predictions based on the latest defined model.
test['Log_Predictions'] = model.predict(test.drop(['Id','Predictions','Log_Predictions'],axis=1)) 
test['Predictions'] = np.exp(test['Log_Predictions']) # Take the exponential of my predictions ready for submission

#test.head()

export = test[['Id','Predictions']].rename({'Predictions':'SalePrice'},axis=1)
export.head()

#export.to_csv(r'C:\Users\sjenkins\OneDrive - Dunelm (Soft Furnishings) Ltd\Documents\Side_Projects\House_Price_Regression\submission.csv',index='Id')

Summary¶

I'm really happy with what I've learnt in this project. Reading and making notes from other notebooks on Kaggle has been inspiring.

I'm also midway through Andrew Ng's Machine Learning Course, and have been able to apply my newly learnt understanding of Bias/Variance and Regularistation to the problem.

Other Avenues To Try (Inspired by other notebooks):

Building a neural network
Applying ensemble methods to aggregate my models.

To Further Practice Bias/Variance Understanding

Experiment with the number of features in the model, using feature importance to decide which to add in first: https://machinelearningmastery.com/calculate-feature-importance-with-python/
Experiment with training set size (Learning Curves: See Week 6 of Andrew Ng's Course), random selection of properties.

Bibliography (Notebooks for inspiration):

END

	Id	MSSubClass	MSZoning	LotFrontage	LotArea	Street	Alley	LotShape	LandContour	Utilities	LotConfig	LandSlope	Neighborhood	Condition1	Condition2	BldgType	HouseStyle	OverallQual	OverallCond	YearBuilt	YearRemodAdd	RoofStyle	RoofMatl	Exterior1st	Exterior2nd	MasVnrType	MasVnrArea	ExterQual	ExterCond	Foundation	BsmtQual	BsmtCond	BsmtExposure	BsmtFinType1	BsmtFinSF1	BsmtFinType2	BsmtUnfSF	TotalBsmtSF	Heating	HeatingQC	CentralAir	Electrical	1stFlrSF	2ndFlrSF	GrLivArea	BsmtFullBath	BsmtHalfBath	FullBath	HalfBath	BedroomAbvGr	KitchenAbvGr	KitchenQual	TotRmsAbvGrd	Functional	Fireplaces	FireplaceQu	GarageType	GarageYrBlt	GarageFinish	GarageCars	GarageArea	GarageQual	GarageCond	PavedDrive	WoodDeckSF	OpenPorchSF	EnclosedPorch	PoolQC	Fence	MiscFeature	MoSold	YrSold	SaleType	SaleCondition	SalePrice
0	1	60	RL	65.0	8450	Pave	NaN	Reg	Lvl	AllPub	Inside	Gtl	CollgCr	Norm	Norm	1Fam	2Story	7	5	2003	2003	Gable	CompShg	VinylSd	VinylSd	BrkFace	196.0	Gd	TA	PConc	Gd	TA	No	GLQ	706	Unf	150	856	GasA	Ex	Y	SBrkr	856	854	1710	1	0	2	1	3	1	Gd	8	Typ	0	NaN	Attchd	2003.0	RFn	2	548	TA	TA	Y	0	61	0	NaN	NaN	NaN	2	2008	WD	Normal	208500
1	2	20	RL	80.0	9600	Pave	NaN	Reg	Lvl	AllPub	FR2	Gtl	Veenker	Feedr	Norm	1Fam	1Story	6	8	1976	1976	Gable	CompShg	MetalSd	MetalSd	None	0.0	TA	TA	CBlock	Gd	TA	Gd	ALQ	978	Unf	284	1262	GasA	Ex	Y	SBrkr	1262	0	1262	0	1	2	0	3	1	TA	6	Typ	1	TA	Attchd	1976.0	RFn	2	460	TA	TA	Y	298	0	0	NaN	NaN	NaN	5	2007	WD	Normal	181500
2	3	60	RL	68.0	11250	Pave	NaN	IR1	Lvl	AllPub	Inside	Gtl	CollgCr	Norm	Norm	1Fam	2Story	7	5	2001	2002	Gable	CompShg	VinylSd	VinylSd	BrkFace	162.0	Gd	TA	PConc	Gd	TA	Mn	GLQ	486	Unf	434	920	GasA	Ex	Y	SBrkr	920	866	1786	1	0	2	1	3	1	Gd	6	Typ	1	TA	Attchd	2001.0	RFn	2	608	TA	TA	Y	0	42	0	NaN	NaN	NaN	9	2008	WD	Normal	223500
3	4	70	RL	60.0	9550	Pave	NaN	IR1	Lvl	AllPub	Corner	Gtl	Crawfor	Norm	Norm	1Fam	2Story	7	5	1915	1970	Gable	CompShg	Wd Sdng	Wd Shng	None	0.0	TA	TA	BrkTil	TA	Gd	No	ALQ	216	Unf	540	756	GasA	Gd	Y	SBrkr	961	756	1717	1	0	1	0	3	1	Gd	7	Typ	1	Gd	Detchd	1998.0	Unf	3	642	TA	TA	Y	0	35	272	NaN	NaN	NaN	2	2006	WD	Abnorml	140000
4	5	60	RL	84.0	14260	Pave	NaN	IR1	Lvl	AllPub	FR2	Gtl	NoRidge	Norm	Norm	1Fam	2Story	8	5	2000	2000	Gable	CompShg	VinylSd	VinylSd	BrkFace	350.0	Gd	TA	PConc	Gd	TA	Av	GLQ	655	Unf	490	1145	GasA	Ex	Y	SBrkr	1145	1053	2198	1	0	2	1	4	1	Gd	9	Typ	1	TA	Attchd	2000.0	RFn	3	836	TA	TA	Y	192	84	0	NaN	NaN	NaN	12	2008	WD	Normal	250000

	Id	MSSubClass	MSZoning	LotFrontage	LotArea	Street	Alley	LotShape	LandContour	Utilities	LotConfig	LandSlope	Neighborhood	Condition1	Condition2	BldgType	HouseStyle	OverallQual	OverallCond	YearBuilt	YearRemodAdd	RoofStyle	RoofMatl	Exterior1st	Exterior2nd	MasVnrType	MasVnrArea	ExterQual	ExterCond	Foundation	BsmtQual	BsmtCond	BsmtExposure	BsmtFinType1	BsmtFinSF1	BsmtFinType2	BsmtFinSF2	BsmtUnfSF	TotalBsmtSF	Heating	HeatingQC	CentralAir	Electrical	1stFlrSF	2ndFlrSF	GrLivArea	FullBath	HalfBath	BedroomAbvGr	KitchenAbvGr	KitchenQual	TotRmsAbvGrd	Functional	Fireplaces	FireplaceQu	GarageType	GarageYrBlt	GarageFinish	GarageCars	GarageArea	GarageQual	GarageCond	PavedDrive	WoodDeckSF	OpenPorchSF	ScreenPorch	PoolQC	Fence	MiscFeature	MiscVal	MoSold	YrSold	SaleType	SaleCondition
0	1461	20	RH	80.0	11622	Pave	NaN	Reg	Lvl	AllPub	Inside	Gtl	NAmes	Feedr	Norm	1Fam	1Story	5	6	1961	1961	Gable	CompShg	VinylSd	VinylSd	None	0.0	TA	TA	CBlock	TA	TA	No	Rec	468.0	LwQ	144.0	270.0	882.0	GasA	TA	Y	SBrkr	896	0	896	1	0	2	1	TA	5	Typ	0	NaN	Attchd	1961.0	Unf	1.0	730.0	TA	TA	Y	140	0	120	NaN	MnPrv	NaN	0	6	2010	WD	Normal
1	1462	20	RL	81.0	14267	Pave	NaN	IR1	Lvl	AllPub	Corner	Gtl	NAmes	Norm	Norm	1Fam	1Story	6	6	1958	1958	Hip	CompShg	Wd Sdng	Wd Sdng	BrkFace	108.0	TA	TA	CBlock	TA	TA	No	ALQ	923.0	Unf	0.0	406.0	1329.0	GasA	TA	Y	SBrkr	1329	0	1329	1	1	3	1	Gd	6	Typ	0	NaN	Attchd	1958.0	Unf	1.0	312.0	TA	TA	Y	393	36	0	NaN	NaN	Gar2	12500	6	2010	WD	Normal
2	1463	60	RL	74.0	13830	Pave	NaN	IR1	Lvl	AllPub	Inside	Gtl	Gilbert	Norm	Norm	1Fam	2Story	5	5	1997	1998	Gable	CompShg	VinylSd	VinylSd	None	0.0	TA	TA	PConc	Gd	TA	No	GLQ	791.0	Unf	0.0	137.0	928.0	GasA	Gd	Y	SBrkr	928	701	1629	2	1	3	1	TA	6	Typ	1	TA	Attchd	1997.0	Fin	2.0	482.0	TA	TA	Y	212	34	0	NaN	MnPrv	NaN	0	3	2010	WD	Normal
3	1464	60	RL	78.0	9978	Pave	NaN	IR1	Lvl	AllPub	Inside	Gtl	Gilbert	Norm	Norm	1Fam	2Story	6	6	1998	1998	Gable	CompShg	VinylSd	VinylSd	BrkFace	20.0	TA	TA	PConc	TA	TA	No	GLQ	602.0	Unf	0.0	324.0	926.0	GasA	Ex	Y	SBrkr	926	678	1604	2	1	3	1	Gd	7	Typ	1	Gd	Attchd	1998.0	Fin	2.0	470.0	TA	TA	Y	360	36	0	NaN	NaN	NaN	0	6	2010	WD	Normal
4	1465	120	RL	43.0	5005	Pave	NaN	IR1	HLS	AllPub	Inside	Gtl	StoneBr	Norm	Norm	TwnhsE	1Story	8	5	1992	1992	Gable	CompShg	HdBoard	HdBoard	None	0.0	Gd	TA	PConc	Gd	TA	No	ALQ	263.0	Unf	0.0	1017.0	1280.0	GasA	Ex	Y	SBrkr	1280	0	1280	2	0	2	1	Gd	5	Typ	0	NaN	Attchd	1992.0	RFn	2.0	506.0	TA	TA	Y	0	82	144	NaN	NaN	NaN	0	1	2010	WD	Normal

	count	mean	std	min	25%	50%	75%	max
Id	1460.0	730.500000	421.610009	1.0	365.75	730.5	1095.25	1460.0
MSSubClass	1460.0	56.897260	42.300571	20.0	20.00	50.0	70.00	190.0
LotFrontage	1201.0	70.049958	24.284752	21.0	59.00	69.0	80.00	313.0
LotArea	1460.0	10516.828082	9981.264932	1300.0	7553.50	9478.5	11601.50	215245.0
OverallQual	1460.0	6.099315	1.382997	1.0	5.00	6.0	7.00	10.0
OverallCond	1460.0	5.575342	1.112799	1.0	5.00	5.0	6.00	9.0
YearBuilt	1460.0	1971.267808	30.202904	1872.0	1954.00	1973.0	2000.00	2010.0
YearRemodAdd	1460.0	1984.865753	20.645407	1950.0	1967.00	1994.0	2004.00	2010.0
MasVnrArea	1452.0	103.685262	181.066207	0.0	0.00	0.0	166.00	1600.0
BsmtFinSF1	1460.0	443.639726	456.098091	0.0	0.00	383.5	712.25	5644.0
BsmtFinSF2	1460.0	46.549315	161.319273	0.0	0.00	0.0	0.00	1474.0
BsmtUnfSF	1460.0	567.240411	441.866955	0.0	223.00	477.5	808.00	2336.0
TotalBsmtSF	1460.0	1057.429452	438.705324	0.0	795.75	991.5	1298.25	6110.0
1stFlrSF	1460.0	1162.626712	386.587738	334.0	882.00	1087.0	1391.25	4692.0
2ndFlrSF	1460.0	346.992466	436.528436	0.0	0.00	0.0	728.00	2065.0
LowQualFinSF	1460.0	5.844521	48.623081	0.0	0.00	0.0	0.00	572.0
GrLivArea	1460.0	1515.463699	525.480383	334.0	1129.50	1464.0	1776.75	5642.0
BsmtFullBath	1460.0	0.425342	0.518911	0.0	0.00	0.0	1.00	3.0
BsmtHalfBath	1460.0	0.057534	0.238753	0.0	0.00	0.0	0.00	2.0
FullBath	1460.0	1.565068	0.550916	0.0	1.00	2.0	2.00	3.0
HalfBath	1460.0	0.382877	0.502885	0.0	0.00	0.0	1.00	2.0
BedroomAbvGr	1460.0	2.866438	0.815778	0.0	2.00	3.0	3.00	8.0
KitchenAbvGr	1460.0	1.046575	0.220338	0.0	1.00	1.0	1.00	3.0
TotRmsAbvGrd	1460.0	6.517808	1.625393	2.0	5.00	6.0	7.00	14.0
Fireplaces	1460.0	0.613014	0.644666	0.0	0.00	1.0	1.00	3.0
GarageYrBlt	1379.0	1978.506164	24.689725	1900.0	1961.00	1980.0	2002.00	2010.0
GarageCars	1460.0	1.767123	0.747315	0.0	1.00	2.0	2.00	4.0
GarageArea	1460.0	472.980137	213.804841	0.0	334.50	480.0	576.00	1418.0
WoodDeckSF	1460.0	94.244521	125.338794	0.0	0.00	0.0	168.00	857.0
OpenPorchSF	1460.0	46.660274	66.256028	0.0	0.00	25.0	68.00	547.0
EnclosedPorch	1460.0	21.954110	61.119149	0.0	0.00	0.0	0.00	552.0
3SsnPorch	1460.0	3.409589	29.317331	0.0	0.00	0.0	0.00	508.0
ScreenPorch	1460.0	15.060959	55.757415	0.0	0.00	0.0	0.00	480.0
PoolArea	1460.0	2.758904	40.177307	0.0	0.00	0.0	0.00	738.0
MiscVal	1460.0	43.489041	496.123024	0.0	0.00	0.0	0.00	15500.0
MoSold	1460.0	6.321918	2.703626	1.0	5.00	6.0	8.00	12.0
YrSold	1460.0	2007.815753	1.328095	2006.0	2007.00	2008.0	2009.00	2010.0
SalePrice	1460.0	180921.195890	79442.502883	34900.0	129975.00	163000.0	214000.00	755000.0

	Id	MSSubClass	MSZoning	LotFrontage	LotArea	Street	Alley	LotShape	LandContour	Utilities	LotConfig	LandSlope	Neighborhood	Condition1	Condition2	BldgType	HouseStyle	OverallQual	OverallCond	YearBuilt	YearRemodAdd	RoofStyle	RoofMatl	Exterior1st	Exterior2nd	MasVnrType	MasVnrArea	ExterQual	ExterCond	Foundation	BsmtQual	BsmtCond	BsmtExposure	BsmtFinType1	BsmtFinSF1	BsmtFinType2	BsmtFinSF2	BsmtUnfSF	TotalBsmtSF	Heating	HeatingQC	CentralAir	Electrical	1stFlrSF	2ndFlrSF	LowQualFinSF	GrLivArea	BsmtFullBath	BsmtHalfBath	FullBath	HalfBath	BedroomAbvGr	KitchenAbvGr	KitchenQual	TotRmsAbvGrd	Functional	Fireplaces	FireplaceQu	GarageType	GarageYrBlt	GarageFinish	GarageCars	GarageArea	GarageQual	GarageCond	PavedDrive	WoodDeckSF	OpenPorchSF	EnclosedPorch	3SsnPorch	ScreenPorch	PoolArea	PoolQC	Fence	MiscFeature	MiscVal	MoSold	YrSold	SaleType	SaleCondition	SalePrice	LogSalePrice
0	1	60	RL	65.0	8450	Pave	NaN	Reg	Lvl	AllPub	Inside	Gtl	CollgCr	Norm	Norm	1Fam	2Story	7	5	2003	2003	Gable	CompShg	VinylSd	VinylSd	BrkFace	196.0	Gd	TA	PConc	Gd	TA	No	GLQ	706.0	Unf	0.0	150.0	856.0	GasA	Ex	Y	SBrkr	856	854	0	1710	1.0	0.0	2	1	3	1	Gd	8	Typ	0	NaN	Attchd	2003.0	RFn	2.0	548.0	TA	TA	Y	0	61	0	0	0	0	NaN	NaN	NaN	0	2	2008	WD	Normal	208500.0	12.247694
1	2	20	RL	80.0	9600	Pave	NaN	Reg	Lvl	AllPub	FR2	Gtl	Veenker	Feedr	Norm	1Fam	1Story	6	8	1976	1976	Gable	CompShg	MetalSd	MetalSd	None	0.0	TA	TA	CBlock	Gd	TA	Gd	ALQ	978.0	Unf	0.0	284.0	1262.0	GasA	Ex	Y	SBrkr	1262	0	0	1262	0.0	1.0	2	0	3	1	TA	6	Typ	1	TA	Attchd	1976.0	RFn	2.0	460.0	TA	TA	Y	298	0	0	0	0	0	NaN	NaN	NaN	0	5	2007	WD	Normal	181500.0	12.109011
2	3	60	RL	68.0	11250	Pave	NaN	IR1	Lvl	AllPub	Inside	Gtl	CollgCr	Norm	Norm	1Fam	2Story	7	5	2001	2002	Gable	CompShg	VinylSd	VinylSd	BrkFace	162.0	Gd	TA	PConc	Gd	TA	Mn	GLQ	486.0	Unf	0.0	434.0	920.0	GasA	Ex	Y	SBrkr	920	866	0	1786	1.0	0.0	2	1	3	1	Gd	6	Typ	1	TA	Attchd	2001.0	RFn	2.0	608.0	TA	TA	Y	0	42	0	0	0	0	NaN	NaN	NaN	0	9	2008	WD	Normal	223500.0	12.317167
3	4	70	RL	60.0	9550	Pave	NaN	IR1	Lvl	AllPub	Corner	Gtl	Crawfor	Norm	Norm	1Fam	2Story	7	5	1915	1970	Gable	CompShg	Wd Sdng	Wd Shng	None	0.0	TA	TA	BrkTil	TA	Gd	No	ALQ	216.0	Unf	0.0	540.0	756.0	GasA	Gd	Y	SBrkr	961	756	0	1717	1.0	0.0	1	0	3	1	Gd	7	Typ	1	Gd	Detchd	1998.0	Unf	3.0	642.0	TA	TA	Y	0	35	272	0	0	0	NaN	NaN	NaN	0	2	2006	WD	Abnorml	140000.0	11.849398
4	5	60	RL	84.0	14260	Pave	NaN	IR1	Lvl	AllPub	FR2	Gtl	NoRidge	Norm	Norm	1Fam	2Story	8	5	2000	2000	Gable	CompShg	VinylSd	VinylSd	BrkFace	350.0	Gd	TA	PConc	Gd	TA	Av	GLQ	655.0	Unf	0.0	490.0	1145.0	GasA	Ex	Y	SBrkr	1145	1053	0	2198	1.0	0.0	2	1	4	1	Gd	9	Typ	1	TA	Attchd	2000.0	RFn	3.0	836.0	TA	TA	Y	192	84	0	0	0	0	NaN	NaN	NaN	0	12	2008	WD	Normal	250000.0	12.429216
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
2906	2915	160	RM	21.0	1936	Pave	NaN	Reg	Lvl	AllPub	Inside	Gtl	MeadowV	Norm	Norm	Twnhs	2Story	4	7	1970	1970	Gable	CompShg	CemntBd	CmentBd	None	0.0	TA	TA	CBlock	TA	TA	No	Unf	0.0	Unf	0.0	546.0	546.0	GasA	Gd	Y	SBrkr	546	546	0	1092	0.0	0.0	1	1	3	1	TA	5	Typ	0	NaN	NaN	NaN	NaN	0.0	0.0	NaN	NaN	Y	0	0	0	0	0	0	NaN	NaN	NaN	0	6	2006	WD	Normal	NaN	NaN
2907	2916	160	RM	21.0	1894	Pave	NaN	Reg	Lvl	AllPub	Inside	Gtl	MeadowV	Norm	Norm	TwnhsE	2Story	4	5	1970	1970	Gable	CompShg	CemntBd	CmentBd	None	0.0	TA	TA	CBlock	TA	TA	No	Rec	252.0	Unf	0.0	294.0	546.0	GasA	TA	Y	SBrkr	546	546	0	1092	0.0	0.0	1	1	3	1	TA	6	Typ	0	NaN	CarPort	1970.0	Unf	1.0	286.0	TA	TA	Y	0	24	0	0	0	0	NaN	NaN	NaN	0	4	2006	WD	Abnorml	NaN	NaN
2908	2917	20	RL	160.0	20000	Pave	NaN	Reg	Lvl	AllPub	Inside	Gtl	Mitchel	Norm	Norm	1Fam	1Story	5	7	1960	1996	Gable	CompShg	VinylSd	VinylSd	None	0.0	TA	TA	CBlock	TA	TA	No	ALQ	1224.0	Unf	0.0	0.0	1224.0	GasA	Ex	Y	SBrkr	1224	0	0	1224	1.0	0.0	1	0	4	1	TA	7	Typ	1	TA	Detchd	1960.0	Unf	2.0	576.0	TA	TA	Y	474	0	0	0	0	0	NaN	NaN	NaN	0	9	2006	WD	Abnorml	NaN	NaN
2909	2918	85	RL	62.0	10441	Pave	NaN	Reg	Lvl	AllPub	Inside	Gtl	Mitchel	Norm	Norm	1Fam	SFoyer	5	5	1992	1992	Gable	CompShg	HdBoard	Wd Shng	None	0.0	TA	TA	PConc	Gd	TA	Av	GLQ	337.0	Unf	0.0	575.0	912.0	GasA	TA	Y	SBrkr	970	0	0	970	0.0	1.0	1	0	3	1	TA	6	Typ	0	NaN	NaN	NaN	NaN	0.0	0.0	NaN	NaN	Y	80	32	0	0	0	0	NaN	MnPrv	Shed	700	7	2006	WD	Normal	NaN	NaN
2910	2919	60	RL	74.0	9627	Pave	NaN	Reg	Lvl	AllPub	Inside	Mod	Mitchel	Norm	Norm	1Fam	2Story	7	5	1993	1994	Gable	CompShg	HdBoard	HdBoard	BrkFace	94.0	TA	TA	PConc	Gd	TA	Av	LwQ	758.0	Unf	0.0	238.0	996.0	GasA	Ex	Y	SBrkr	996	1004	0	2000	0.0	0.0	2	1	3	1	TA	9	Typ	1	TA	Attchd	1993.0	Fin	3.0	650.0	TA	TA	Y	190	48	0	0	0	0	NaN	NaN	NaN	0	11	2006	WD	Normal	NaN	NaN

	LogSalePrice	Model_Output(Log)
1043	11.349229	11.413496
481	11.898188	11.840372
886	11.947949	11.947662
54	11.775290	11.723879
968	12.028739	12.134094

	Feature	Missing Value Count	Missing Value %
72	PoolQC	2902	0.996908
74	MiscFeature	2808	0.964617
6	Alley	2713	0.931982
73	Fence	2340	0.803847
57	FireplaceQu	1420	0.487805
3	LotFrontage	482	0.165579
59	GarageYrBlt	159	0.054620
60	GarageFinish	159	0.054620
63	GarageQual	159	0.054620
64	GarageCond	159	0.054620
58	GarageType	157	0.053933
32	BsmtExposure	82	0.028169
31	BsmtCond	82	0.028169
30	BsmtQual	81	0.027825
35	BsmtFinType2	80	0.027482
33	BsmtFinType1	79	0.027138
25	MasVnrType	24	0.008245
26	MasVnrArea	23	0.007901
2	MSZoning	4	0.001374
55	Functional	2	0.000687
48	BsmtHalfBath	2	0.000687
47	BsmtFullBath	2	0.000687
9	Utilities	2	0.000687
61	GarageCars	1	0.000344
53	KitchenQual	1	0.000344
34	BsmtFinSF1	1	0.000344
78	SaleType	1	0.000344
36	BsmtFinSF2	1	0.000344
37	BsmtUnfSF	1	0.000344
38	TotalBsmtSF	1	0.000344
24	Exterior2nd	1	0.000344
23	Exterior1st	1	0.000344
62	GarageArea	1	0.000344
42	Electrical	1	0.000344

	Id	LotFrontage	LotArea	OverallQual	OverallCond	MasVnrArea	BsmtFinSF1	BsmtFinSF2	BsmtUnfSF	TotalBsmtSF	1stFlrSF	2ndFlrSF	LowQualFinSF	GrLivArea	BsmtFullBath	BsmtHalfBath	FullBath	HalfBath	BedroomAbvGr	KitchenAbvGr	TotRmsAbvGrd	Fireplaces	GarageCars	GarageArea	WoodDeckSF	OpenPorchSF	EnclosedPorch	3SsnPorch	ScreenPorch	PoolArea	MiscVal	MSSubClass_150	MSSubClass_160	MSSubClass_180	MSSubClass_190	MSSubClass_20	MSSubClass_30	MSSubClass_40	MSSubClass_45	MSSubClass_50	MSSubClass_60	MSSubClass_70	MSSubClass_75	MSSubClass_80	MSSubClass_85	MSSubClass_90	MSZoning_FV	MSZoning_RH	MSZoning_RL	MSZoning_RM	Street_Pave	Alley_None	Alley_Pave	LotShape_IR2	LotShape_IR3	LotShape_Reg	LandContour_HLS	LandContour_Low	LandContour_Lvl	Utilities_NoSeWa	LotConfig_CulDSac	LotConfig_FR2	LotConfig_FR3	LotConfig_Inside	LandSlope_Mod	LandSlope_Sev	Neighborhood_Blueste	Neighborhood_BrDale	Neighborhood_BrkSide	Neighborhood_ClearCr	Neighborhood_CollgCr	Neighborhood_Crawfor	Neighborhood_Edwards	Neighborhood_Gilbert	Neighborhood_IDOTRR	Neighborhood_MeadowV	Neighborhood_Mitchel	Neighborhood_NAmes	Neighborhood_NPkVill	Neighborhood_NWAmes	Neighborhood_NoRidge	Neighborhood_NridgHt	Neighborhood_OldTown	Neighborhood_SWISU	Neighborhood_Sawyer	Neighborhood_SawyerW	Neighborhood_Somerst	Neighborhood_StoneBr	Neighborhood_Timber	Neighborhood_Veenker	Condition1_Feedr	Condition1_Norm	Condition1_PosA	Condition1_PosN	Condition1_RRAe	Condition1_RRAn	Condition1_RRNe	Condition1_RRNn	Condition2_Feedr	Condition2_Norm	Condition2_PosA	Condition2_PosN	Condition2_RRAe	Condition2_RRAn	Condition2_RRNn	BldgType_2fmCon	BldgType_Duplex	BldgType_Twnhs	BldgType_TwnhsE	HouseStyle_1.5Unf	HouseStyle_1Story	HouseStyle_2.5Fin	HouseStyle_2.5Unf	HouseStyle_2Story	HouseStyle_SFoyer	HouseStyle_SLvl	YearBuilt_1875	YearBuilt_1879	YearBuilt_1880	YearBuilt_1882	YearBuilt_1885	YearBuilt_1890	YearBuilt_1892	YearBuilt_1893	YearBuilt_1895	YearBuilt_1896	YearBuilt_1898	YearBuilt_1900	YearBuilt_1901	YearBuilt_1902	YearBuilt_1904	YearBuilt_1905	YearBuilt_1906	YearBuilt_1907	YearBuilt_1908	YearBuilt_1910	YearBuilt_1911	YearBuilt_1912	YearBuilt_1913	YearBuilt_1914	YearBuilt_1915	YearBuilt_1916	YearBuilt_1917	YearBuilt_1918	YearBuilt_1919	YearBuilt_1920	YearBuilt_1921	YearBuilt_1922	YearBuilt_1923	YearBuilt_1924	YearBuilt_1925	YearBuilt_1926	YearBuilt_1927	YearBuilt_1928	YearBuilt_1929	YearBuilt_1930	YearBuilt_1931	YearBuilt_1932	YearBuilt_1934	YearBuilt_1935	YearBuilt_1936	YearBuilt_1937	YearBuilt_1938	YearBuilt_1939	YearBuilt_1940	YearBuilt_1941	YearBuilt_1942	YearBuilt_1945	YearBuilt_1946	YearBuilt_1947	YearBuilt_1948	YearBuilt_1949	YearBuilt_1950	YearBuilt_1951	YearBuilt_1952	YearBuilt_1953	YearBuilt_1954	YearBuilt_1955	YearBuilt_1956	YearBuilt_1957	YearBuilt_1958	YearBuilt_1959	YearBuilt_1960	YearBuilt_1961	YearBuilt_1962	YearBuilt_1963	YearBuilt_1964	YearBuilt_1965	YearBuilt_1966	YearBuilt_1967	YearBuilt_1968	YearBuilt_1969	YearBuilt_1970	YearBuilt_1971	YearBuilt_1972	YearBuilt_1973	YearBuilt_1974	YearBuilt_1975	YearBuilt_1976	YearBuilt_1977	YearBuilt_1978	YearBuilt_1979	YearBuilt_1980	YearBuilt_1981	YearBuilt_1982	YearBuilt_1983	YearBuilt_1984	YearBuilt_1985	YearBuilt_1986	YearBuilt_1987	YearBuilt_1988	YearBuilt_1989	YearBuilt_1990	YearBuilt_1991	YearBuilt_1992	YearBuilt_1993	YearBuilt_1994	YearBuilt_1995	YearBuilt_1996	YearBuilt_1997	YearBuilt_1998	YearBuilt_1999	YearBuilt_2000	YearBuilt_2001	YearBuilt_2002	YearBuilt_2003	YearBuilt_2004	YearBuilt_2005	YearBuilt_2006	YearBuilt_2007	YearBuilt_2008	YearBuilt_2009	YearBuilt_2010	YearRemodAdd_1951	YearRemodAdd_1952	YearRemodAdd_1953	YearRemodAdd_1954	YearRemodAdd_1955	YearRemodAdd_1956	YearRemodAdd_1957	YearRemodAdd_1958	YearRemodAdd_1959	YearRemodAdd_1960	YearRemodAdd_1961	YearRemodAdd_1962	YearRemodAdd_1963	YearRemodAdd_1964	YearRemodAdd_1965	YearRemodAdd_1966	YearRemodAdd_1967	YearRemodAdd_1968	YearRemodAdd_1969	YearRemodAdd_1970	YearRemodAdd_1971	YearRemodAdd_1972	YearRemodAdd_1973	YearRemodAdd_1974	YearRemodAdd_1975	YearRemodAdd_1976	YearRemodAdd_1977	YearRemodAdd_1978	YearRemodAdd_1979	YearRemodAdd_1980	YearRemodAdd_1981	YearRemodAdd_1982	YearRemodAdd_1983	YearRemodAdd_1984	YearRemodAdd_1985	YearRemodAdd_1986	YearRemodAdd_1987	YearRemodAdd_1988	YearRemodAdd_1989	YearRemodAdd_1990	YearRemodAdd_1991	YearRemodAdd_1992	YearRemodAdd_1993	YearRemodAdd_1994	YearRemodAdd_1995	YearRemodAdd_1996	YearRemodAdd_1997	YearRemodAdd_1998	YearRemodAdd_1999	YearRemodAdd_2000	YearRemodAdd_2001	YearRemodAdd_2002	YearRemodAdd_2003	YearRemodAdd_2004	YearRemodAdd_2005	YearRemodAdd_2006	YearRemodAdd_2007	YearRemodAdd_2008	YearRemodAdd_2009	YearRemodAdd_2010	RoofStyle_Gable	RoofStyle_Gambrel	RoofStyle_Hip	RoofStyle_Mansard	RoofStyle_Shed	RoofMatl_Membran	RoofMatl_Metal	RoofMatl_Roll	RoofMatl_Tar&Grv	RoofMatl_WdShake	RoofMatl_WdShngl	Exterior1st_AsphShn	Exterior1st_BrkComm	Exterior1st_BrkFace	Exterior1st_CBlock	Exterior1st_CemntBd	Exterior1st_HdBoard	Exterior1st_ImStucc	Exterior1st_MetalSd	Exterior1st_Plywood	Exterior1st_Stone	Exterior1st_Stucco	Exterior1st_VinylSd	Exterior1st_Wd Sdng	Exterior1st_WdShing	Exterior2nd_AsphShn	Exterior2nd_Brk Cmn	Exterior2nd_BrkFace	Exterior2nd_CBlock	Exterior2nd_CmentBd	Exterior2nd_HdBoard	Exterior2nd_ImStucc	Exterior2nd_MetalSd	Exterior2nd_Other	Exterior2nd_Plywood	Exterior2nd_Stone	Exterior2nd_Stucco	Exterior2nd_VinylSd	Exterior2nd_Wd Sdng	Exterior2nd_Wd Shng	MasVnrType_BrkFace	MasVnrType_None	MasVnrType_Stone	ExterQual_Fa	ExterQual_Gd	ExterQual_TA	ExterCond_Fa	ExterCond_Gd	ExterCond_Po	ExterCond_TA	Foundation_CBlock	Foundation_PConc	Foundation_Slab	Foundation_Stone	Foundation_Wood	BsmtQual_Fa	BsmtQual_Gd	BsmtQual_None	BsmtQual_TA	BsmtCond_Gd	BsmtCond_None	BsmtCond_Po	BsmtCond_TA	BsmtExposure_Gd	BsmtExposure_Mn	BsmtExposure_No	BsmtExposure_None	BsmtFinType1_BLQ	BsmtFinType1_GLQ	BsmtFinType1_LwQ	BsmtFinType1_None	BsmtFinType1_Rec	BsmtFinType1_Unf	BsmtFinType2_BLQ	BsmtFinType2_GLQ	BsmtFinType2_LwQ	BsmtFinType2_None	BsmtFinType2_Rec	BsmtFinType2_Unf	Heating_GasA	Heating_GasW	Heating_Grav	Heating_OthW	Heating_Wall	HeatingQC_Fa	HeatingQC_Gd	HeatingQC_Po	HeatingQC_TA	CentralAir_Y	Electrical_FuseF	Electrical_FuseP	Electrical_Mix	Electrical_SBrkr	KitchenQual_Fa	KitchenQual_Gd	KitchenQual_TA	Functional_Maj2	Functional_Min1	Functional_Min2	Functional_Mod	Functional_Sev	Functional_Typ	FireplaceQu_Fa	FireplaceQu_Gd	FireplaceQu_None	FireplaceQu_Po	FireplaceQu_TA	GarageType_Attchd	GarageType_Basment	GarageType_BuiltIn	GarageType_CarPort	GarageType_Detchd	GarageType_None	GarageYrBlt_1896.0	GarageYrBlt_1900.0	GarageYrBlt_1906.0	GarageYrBlt_1908.0	GarageYrBlt_1910.0	GarageYrBlt_1914.0	GarageYrBlt_1915.0	GarageYrBlt_1916.0	GarageYrBlt_1917.0	GarageYrBlt_1918.0	GarageYrBlt_1919.0	GarageYrBlt_1920.0	GarageYrBlt_1921.0	GarageYrBlt_1922.0	GarageYrBlt_1923.0	GarageYrBlt_1924.0	GarageYrBlt_1925.0	GarageYrBlt_1926.0	GarageYrBlt_1927.0	GarageYrBlt_1928.0	GarageYrBlt_1929.0	GarageYrBlt_1930.0	GarageYrBlt_1931.0	GarageYrBlt_1932.0	GarageYrBlt_1933.0	GarageYrBlt_1934.0	GarageYrBlt_1935.0	GarageYrBlt_1936.0	GarageYrBlt_1937.0	GarageYrBlt_1938.0	GarageYrBlt_1939.0	GarageYrBlt_1940.0	GarageYrBlt_1941.0	GarageYrBlt_1942.0	GarageYrBlt_1943.0	GarageYrBlt_1945.0	GarageYrBlt_1946.0	GarageYrBlt_1947.0	GarageYrBlt_1948.0	GarageYrBlt_1949.0	GarageYrBlt_1950.0	GarageYrBlt_1951.0	GarageYrBlt_1952.0	GarageYrBlt_1953.0	GarageYrBlt_1954.0	GarageYrBlt_1955.0	GarageYrBlt_1956.0	GarageYrBlt_1957.0	GarageYrBlt_1958.0	GarageYrBlt_1959.0	GarageYrBlt_1960.0	GarageYrBlt_1961.0	GarageYrBlt_1962.0	GarageYrBlt_1963.0	GarageYrBlt_1964.0	GarageYrBlt_1965.0	GarageYrBlt_1966.0	GarageYrBlt_1967.0	GarageYrBlt_1968.0	GarageYrBlt_1969.0	GarageYrBlt_1970.0	GarageYrBlt_1971.0	GarageYrBlt_1972.0	GarageYrBlt_1973.0	GarageYrBlt_1974.0	GarageYrBlt_1975.0	GarageYrBlt_1976.0	GarageYrBlt_1977.0	GarageYrBlt_1978.0	GarageYrBlt_1979.0	GarageYrBlt_1980.0	GarageYrBlt_1981.0	GarageYrBlt_1982.0	GarageYrBlt_1983.0	GarageYrBlt_1984.0	GarageYrBlt_1985.0	GarageYrBlt_1986.0	GarageYrBlt_1987.0	GarageYrBlt_1988.0	GarageYrBlt_1989.0	GarageYrBlt_1990.0	GarageYrBlt_1991.0	GarageYrBlt_1992.0	GarageYrBlt_1993.0	GarageYrBlt_1994.0	GarageYrBlt_1995.0	GarageYrBlt_1996.0	GarageYrBlt_1997.0	GarageYrBlt_1998.0	GarageYrBlt_1999.0	GarageYrBlt_2000.0	GarageYrBlt_2001.0	GarageYrBlt_2002.0	GarageYrBlt_2003.0	GarageYrBlt_2004.0	GarageYrBlt_2005.0	GarageYrBlt_2006.0	GarageYrBlt_2007.0	GarageYrBlt_2008.0	GarageYrBlt_2009.0	GarageYrBlt_2010.0	GarageFinish_None	GarageFinish_RFn	GarageFinish_Unf	GarageQual_Fa	GarageQual_Gd	GarageQual_None	GarageQual_Po	GarageQual_TA	GarageCond_Fa	GarageCond_Gd	GarageCond_None	GarageCond_Po	GarageCond_TA	PavedDrive_P	PavedDrive_Y	PoolQC_Fa	PoolQC_Gd	PoolQC_None	Fence_GdWo	Fence_MnPrv	Fence_MnWw	Fence_None	MiscFeature_None	MiscFeature_Othr	MiscFeature_Shed	MiscFeature_TenC	MoSold_10	MoSold_11	MoSold_12	MoSold_2	MoSold_3	MoSold_4	MoSold_5	MoSold_6	MoSold_7	MoSold_8	MoSold_9	YrSold_2007	YrSold_2008	YrSold_2009	YrSold_2010	SaleType_CWD	SaleType_Con	SaleType_ConLD	SaleType_ConLI	SaleType_ConLw	SaleType_New	SaleType_Oth	SaleType_WD	SaleCondition_AdjLand	SaleCondition_Alloca	SaleCondition_Family	SaleCondition_Normal	SaleCondition_Partial
1452	1461	80.0	11622	5	6	0.0	468.0	144.0	270.0	882.0	896	0	0	896	0.0	0.0	1	0	2	1	5	0	1.0	730.0	140	0	0	0	120	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	1	0	0	1	1	0	0	0	1	0	0	1	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	1	0	0	0	1	0	0	0	1	1	0	0	0	0	0	0	0	1	0	0	0	1	0	0	1	0	0	0	0	0	1	0	0	0	1	0	0	0	1	0	0	0	0	0	0	0	1	1	0	0	0	1	0	0	1	0	0	0	0	0	1	0	0	1	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	1	0	0	0	0	1	0	1	0	0	1	0	1	0	0	1	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	1	0	0	0	0	0	0	0	1	0	0	0	1	0
1453	1462	81.0	14267	6	6	108.0	923.0	0.0	406.0	1329.0	1329	0	0	1329	0.0	0.0	1	1	3	1	6	0	1.0	312.0	393	36	0	0	0	0	12500	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	1	0	1	1	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	1	0	0	0	0	1	0	0	0	1	1	0	0	0	0	0	0	0	1	0	0	0	1	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	1	1	0	0	0	0	0	0	0	1	1	0	0	0	1	0	1	0	0	0	0	0	0	1	0	0	1	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	1	0	0	0	0	1	0	1	0	0	1	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	1	0	0	0	0	0	0	0	1	0	0	0	1	0
1454	1463	74.0	13830	5	5	0.0	791.0	0.0	137.0	928.0	928	701	0	1629	0.0	0.0	2	1	3	1	6	1	2.0	482.0	212	34	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	1	0	1	1	0	0	0	0	0	0	1	0	0	0	0	1	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	1	0	0	0	1	0	0	0	1	0	1	0	0	0	0	1	0	0	0	0	0	1	0	0	1	0	0	1	0	0	0	0	0	0	0	0	0	1	1	0	0	0	0	0	1	0	0	1	0	0	0	1	0	0	1	0	0	0	0	0	1	0	0	0	0	1	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	1	0	1	0	0	1	0	1	0	0	1	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	1	0	0	0	1	0
1455	1464	78.0	9978	6	6	20.0	602.0	0.0	324.0	926.0	926	678	0	1604	0.0	0.0	2	1	3	1	7	1	2.0	470.0	360	36	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	1	0	1	1	0	0	0	0	0	0	1	0	0	0	0	1	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	1	0	0	0	0	1	0	0	0	1	0	1	0	0	0	0	0	0	1	0	0	0	1	0	0	1	0	0	1	0	0	0	0	0	0	0	0	0	1	1	0	0	0	0	0	0	0	0	1	0	0	0	1	0	1	0	0	0	0	0	0	1	0	1	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	1	0	1	0	0	1	0	0	0	1	1	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	1	0	0	0	0	0	0	0	1	0	0	0	1	0
1456	1465	43.0	5005	8	5	0.0	263.0	0.0	1017.0	1280.0	1280	0	0	1280	0.0	0.0	2	0	2	1	5	0	2.0	506.0	0	82	0	0	144	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	1	1	0	0	0	0	1	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	1	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	1	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	1	0	0	1	0	0	0	0	1	0	1	0	0	0	0	1	0	0	0	0	0	1	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	1	1	0	0	0	0	0	0	0	0	1	0	0	0	1	0	1	0	0	0	0	0	0	1	0	0	1	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	1	0	0	0	0	1	0	1	0	0	1	0	0	0	1	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	1	0	0	0	1	0
...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...	...
2906	2915	21.0	1936	4	7	0.0	0.0	0.0	546.0	546.0	546	546	0	1092	0.0	0.0	1	1	3	1	5	0	0.0	0.0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	1	1	0	0	0	1	0	0	1	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	1	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	1	0	0	0	1	1	0	0	0	0	0	0	0	1	0	0	0	1	0	0	1	0	0	0	0	0	0	1	0	0	0	0	0	1	1	0	0	0	0	0	1	0	0	1	0	0	0	1	0	0	1	0	0	0	0	0	1	0	0	1	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	1	0	0	0	0	1	0	0	0	0	1	0	0	0	1	0	0	1	0	0	0	1	1	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	1	0
2907	2916	21.0	1894	4	5	0.0	252.0	0.0	294.0	546.0	546	546	0	1092	0.0	0.0	1	1	3	1	6	0	1.0	286.0	0	24	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	1	1	0	0	0	1	0	0	1	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	1	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	1	0	0	0	1	1	0	0	0	0	0	0	0	1	0	0	0	1	0	0	1	0	0	0	0	0	1	0	0	0	0	0	0	1	1	0	0	0	0	0	0	0	1	1	0	0	0	1	0	0	1	0	0	0	0	0	1	0	0	1	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	1	0	0	0	0	1	0	1	0	0	1	0	0	0	1	1	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0
2908	2917	160.0	20000	5	7	0.0	1224.0	0.0	0.0	1224.0	1224	0	0	1224	1.0	0.0	1	0	4	1	7	1	2.0	576.0	474	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	1	0	1	1	0	0	0	1	0	0	1	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	1	0	0	0	1	0	0	0	1	1	0	0	0	0	0	0	0	1	0	0	0	1	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	1	1	0	0	0	0	0	0	0	0	1	0	0	0	1	0	0	1	0	0	0	0	0	1	0	0	0	0	1	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	1	0	0	0	0	1	0	1	0	0	1	0	0	0	1	1	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0
2909	2918	62.0	10441	5	5	0.0	337.0	0.0	575.0	912.0	970	0	0	970	0.0	1.0	1	0	3	1	6	0	0.0	0.0	80	32	0	0	0	0	700	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	1	0	1	1	0	0	0	1	0	0	1	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	1	0	0	0	1	0	0	0	1	0	1	0	0	0	0	1	0	0	0	0	0	1	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	1	1	0	0	0	0	0	0	0	1	1	0	0	0	1	0	0	1	0	0	0	0	0	1	0	0	1	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	1	0	0	0	0	1	0	0	0	0	1	0	0	0	1	0	0	1	0	1	0	0	0	0	1	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	1	0
2910	2919	74.0	9627	7	5	94.0	758.0	0.0	238.0	996.0	996	1004	0	2000	0.0	0.0	2	1	3	1	9	1	3.0	650.0	190	48	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	1	0	1	1	0	0	0	1	0	0	1	0	0	0	0	1	1	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	1	0	0	0	0	1	0	0	0	1	0	1	0	0	0	0	1	0	0	0	0	0	1	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	1	1	0	0	0	0	0	0	0	0	1	0	0	0	1	0	0	1	0	0	0	0	0	1	0	0	0	0	1	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	0	1	0	1	0	0	1	0	0	0	1	1	0	0	0	0	1	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	0	1	0	0	0	1	0

	LogSalePrice	Model_Predictions(Log)
1036	12.185870	1.220246e+01
1124	11.813030	-6.826064e+10
997	11.827043	5.163003e+09
1316	11.320554	1.176862e+01
529	12.089539	1.209618e+01
...	...	...
514	12.259613	1.229887e+01
722	11.608236	1.219272e+01
841	11.801857	1.174449e+01
1079	11.898188	1.195797e+01
1377	11.561716	1.167438e+01

	LogSalePrice	Model_Output(Log)
1043	11.349229	11.430122
481	11.898188	11.859816
886	11.947949	11.928537
54	11.775290	11.733689
968	12.028739	12.091075

	LogSalePrice	Model_Predictions(Log)
1036	12.185870	12.176597
1124	11.813030	12.103738
997	11.827043	11.919396
1316	11.320554	11.545763
529	12.089539	12.119511
...	...	...
514	12.259613	12.277315
722	11.608236	12.073765
841	11.801857	11.755502
1079	11.898188	11.928691
1377	11.561716	11.531173

	Alpha	Training_Errors	Validation_Errors
0	0.01	0.067638	0.158029
1	0.02	0.067900	0.154480
2	0.04	0.068310	0.150081
3	0.08	0.068958	0.144893
4	0.16	0.070011	0.139286
5	0.32	0.071664	0.133808
6	0.64	0.074068	0.128850
7	1.28	0.077259	0.124486
8	2.56	0.081099	0.120626
9	5.12	0.085350	0.117299
10	10.24	0.089893	0.114786
11	20.48	0.094800	0.113502
12	40.96	0.100217	0.113801
13	81.92	0.106172	0.115865
14	163.84	0.112611	0.119703
15	327.68	0.119816	0.125356
16	655.36	0.128788	0.133197
17	1310.00	0.140655	0.143726
18	2621.00	0.154777	0.156220
19	5242.00	0.168191	0.168116
20	10485.00	0.178345	0.177166

	Alpha	Training_Errors	Validation_Errors
0	0.00010	0.080746	0.119756
1	0.00025	0.091350	0.115464
2	0.00050	0.099312	0.114730
3	0.00070	0.102318	0.114789
4	0.00100	0.105630	0.114773
5	0.00250	0.116732	0.120109
6	0.00500	0.126395	0.127217
7	0.01000	0.138529	0.137425
8	0.02000	0.150356	0.146651

	Alpha	Ratio	Training_Errors	Validation_Errors
54	0.0050	0.1	0.100781	0.114168
46	0.0025	0.2	0.099989	0.114393
40	0.0010	0.5	0.099487	0.114611
41	0.0010	0.6	0.101128	0.114618
34	0.0007	0.8	0.100416	0.114651
...	...	...	...	...
94	0.1000	0.5	0.162527	0.159087
95	0.1000	0.6	0.166182	0.162912
96	0.1000	0.7	0.170404	0.167303
97	0.1000	0.8	0.175191	0.172259
98	0.1000	0.9	0.180485	0.177693

	Id	SalePrice
1452	1461	121472.388615
1453	1462	153338.843256
1454	1463	180796.158235
1455	1464	198507.705302
1456	1465	192062.391242